Screen Shot 2016-08-08 at 11.58.53 PM

Homelab: Goals

I’ve been meaning to do a write up on designing my homelab. In my last job, I had access to hardware and some essential networking bits, but now that I’m a Field SE, i’m in a different situation. I have access to internal tools and nested deployments (otherwise called PODs), as well as some Hands-On-Labs deployments. These are great for doing quick demo’s but for continued education purposes, the consensus among SE’s is that “Nothing beats a homelab”.

Keep Reading

Professional Growth: Part 4 – Future Proofing

Intro text:

In keeping with the continued theme of this series, I wanted to know what the responders felt was a good technology path of choice, based on how the market is changing now and what areas they thought had the most potential. I know that the title of this post makes it seem like there is a specific technology horse to bet on at the races, but that isn’t the case. IT is broad and ever changing. These are the number of ideas people think, based on different perspectives within the market.

 

So the next question asked was “What areas of technology do you see the most growth potential that people aren’t considering?” The idea behind this is where can we fill the void, and what emerging technologies are people not hearing about just yet that could become the next buzzword of the tech conference world? Because you know we are all so very ready for the next IoT, Cloud, Hyper-Converged buzzword.

 

Considering I was looking to for a technology, I was surprised that half of the responses still centered on soft-skills. I think we’re seeing a trend with this, no? Granted there are some technology buzzwords that popped up like DevOps and Automation, that was only half of what came up.

So lets get the obvious stuff out of the way, shall we? DevOps. It has many meanings, but I think I like AgileAdmin’s best:

DevOps is the practice of operations and development engineers participating together in the entire service lifecycle, from design through the development process to production support.

The idea that greater collaboration between the two teams can make applications more efficient, set expectations a bit closer to reality and create a more agile group of people who support the business. The key thing being speed. Faster, without the compromise of quality. Now given those two objectives, that likely means its more expensive! You know, the IT Holy Triangle “Fast, Good, Cheap, Pick Two”. Now in the case of today’s technologies, there isn’t a company or IT shop that hasn’t looked at or considered things like Chef, Puppet, Ansible or things like them. These are systems to help take applications from development to production as fast as possible, and are helping shape how we can make our applications more scalable. Which is HUGE!

Automation, the next on the list. Although most people think it falls into DevOps, thats not always the case. It depends on the situation and the business. The fact still remains though, if you work in technology and you allow yourself to do a task more than twice without trying to automate it, you aren’t doing the job justice. Sure scripting is hard, but professional growth isn’t meant to come easily and nothing easy is ever really worth it.

That kicks nicely into the next one, Scripting and Programming. This is actually becoming a focus for many institutions at the elementary level. Which is great to see so much focus on helping our younger ones become effective analytical thinkers! Programming and Scripting gives people a new perspective on problem solving. It helps people realize how to break down processes and find the issue. Even if you don’t work on something technical, learning to code can help you break down a workflow to find inefficiencies and that can be huge! Beyond that, we need to focus on building applications that scale and deploy in a more “Cloud-like” manner. Thus promoting business resiliency! That is the new focus in programming/scripting, because no one likes snowflakes anymore.

Software Defined Anything. From the infrastructure standpoint, this is less about trying to keep up with trends, and more to do with understanding new ways to support demand. One publication had an interesting bit for Networking guys. They posited that Network Engineers not trying to understand SDN technologies would mark them as Dinosaurs waiting for extinction. Not to say that I fully agree, but Architects in the Enterprise space should at least try to understand to see if it makes sense for their business, else they haven’t done a good enough job vetting out technologies (Which is the basis for having an Architect).

Security, big surprise right? This is actually a major issue given the high number of breaches that have been going on. Beyond that, you have to support the businesses compliance efforts. Ever sat in a audit meeting? They are boring and long. The idea being that someone is looking for potential vulnerabilities in your configurations and deployments that don’t meet a standard. No one likes them, but they are important to your customers, so hop to it, get on that security bandwagon. Who knows, maybe you’ll get to go to the blackout conference. I hear its crazy!

So what about the soft-skills? Well, one of my favorite responses was “you need to get out of your comfort zone to grow”, which is so true and not just for the obvious reasons. Ever heard of the Full Stack Engineer? Its a person who can talk about everything from the Infrastructure to the Applications. Understanding how to best support the different technologies that use the infrastructure and tuning everything in business to work well together. Its a hard concept to make a reality, but the easiest thing to start with is getting out of your comfort zone, and learning the ancillary technologies to the one you specialize in. If you cover compute, check out application and networking. If storage, how to work with the applications and network guys to decrease I/O but maintain or increase performance.

Business Skills! This one hits a bit close because its something I talk about with customers often. IT is made up of technology that consistently costs a lot of money. The costs have also continued to grow. People working in IT because of this are constantly thought of as a cost center, instead of a business enabler. Remember when trying new technologies was about making the business more efficient? Yeah, thats mostly forgotten now. In most settings, there are the expected technologies and everything after that is gravy, even if its about making the job of the IT person more efficient or just easier to save them time. The more that you can align your initiatives with the business issues, there more you produce a value. Most companies want automation to lower IT head count, while the IT organization wants automation to increase efficiency. The key is showing how certain actions or systems can reclaim your time, so you spend less on keeping the lights on and more time thinking ahead to better the business.

Cross Department learning. Now this adds on to the business skills portion, because IT needs to understand what others are doing. If we don’t try and spend a little time getting to know the people we support and the things they do, how can we expect to find better solutions for them? We have to become a stakeholder in their objectives and goals. Again, this fits into working with other tiers in the IT stack, the better we understand the parts we support, the more efficient the IT systems and their deployments can get (Not to harp on that again, but its important). You have to work together, if not, you’ll only be seen as a road block, but remember its a two way collaboration so they (the other team) need to buy in as well.

To quote “The First 20 Million is Always the Hardest”, the technologies or soft-skills should support the following idea:

Simplify, Clarify, Economize

Learning about systems or technology that can simplify and automate, Clarify the intent of technology in support of and aligning with the business, Economize by concentrating on technology that helps save the company money, or makes them more efficient at engaging with and handling more business. I think that sums it up pretty well! Plus I got to use a quote from a movie I love.

VMworld First Timers!

First off, I am still alive, just acclimating to my new role and the travel with VMware. To get back into the swing of things and as I get ready for some new things coming up, I thought this would be a good return to writing post.

So for those of you preparing to go to your first VMworld, CONGRATS! You are headed to an excellent conference with roughly 27,000 other people! Yes, let that sink in for half a second… twenty. seven. thousand. people. And that is if they didn’t get additional people to show up this year!

So, for you first timers, I know how you feel. I was you last year, and these are the tips that I got and learned myself that I want to share.

  • SHOES: Get a pair of shoes that are comfortable for walking… 10-15 thousands steps a day in. If you don’t have a step tracker, then you should get one. If you decide to buy a new pair of shoes, BREAK THEM IN FIRST! That was my mistake, I didn’t.
  • SESSIONS: You won’t make them all, it just doesn’t happen. You can try and plan the perfect schedule full of amazing sessions. You’ll want to be there for them all, and then, you’ll walk into the Solutions Exchange. You’ll walk out and realize you missed half the day. Don’t feel bad, it happens to everyone.
  • EVENTS: Lets call it what it is, parties. Call your partners, VAR’s and Vendors and see who has an event going on. This is a great chance to relax, meet people and just get off your aching feet for 30 minutes to an hour. (Because there are about 10 parties happening each night)
  • GOING ALONE?: I did this last year, and although you are surrounded by 27,000 people, there is no weirder way to feel alone. You don’t have to feel that way though. Have you ever met someone that you can talk shop with? Talk about technology and you just go on and on for an hour or two, before you realize time has passed? Yeah, you have 27,000 other people there just like that. This is my single biggest piece of advice. At times you will be walking with a large crowd in the same direction, looking for a table to sit at and see a single chair at a table of 5 or maybe sitting at an event. The best thing you can do at VMworld when you feel alone, or anti-social, just turn to your left or right and introduce yourself. You will be amazed at the people you meet. Heck, a couple times the people I met were VMworld presenters!
  • KEYNOTE: Although its great to attend the main keynote speech in the main hall, there are additional keynotes each day. Instead of going to the main hall, go to the community space. Sit with the bloggers, check out vBrownBag, talk with the VMware Engineers at the “Office of the CTO” Booth (VERY COOL FUTURE TECH!). Get out of the main room and the absolutely insane crowd. Oh and don’t expect to have great mobile data service during this time.
  • COMMUNITY: This is the real reason people go. At my last job, my coworker helped persuade my boss to send me to VMworld (THANK YOU!!) and this was his tip. Meeting with and networking in the community is so much more valuable than just attending the sessions (which are recorded and put on youtube). He was so very right. The people that I met at events, while walking towards a session and even in the community lounge I’ve continued to talk with via twitter and slack. Amazing people!

Thats it for now, but for those heading to their first VMworld, enjoy it! Its a great experience and I recommend hitting up all of the community based parties and events, those were my favorite.

home-hero-background

Ramping up at VMware

I’m on week 3 at VMware, working my way through training and tasks designed to get me “Ramped Up” in the role. Its been incredible so far and for so many more reasons than I expected. First, let me say that I’m truly sorry I haven’t done the next Professional Growth post, I’m actually going to combine question 4 into post 3, and do a repost so look out for that. I’ve been a little busy but I promise to get back on track with those.

First, lets get this question out of the way.

“You went to VMware amidst the Dell Acquisition? Why?”

Yes, this was a major concern even before getting a call to do the first interview. There are so many articles and blog posts centered around VMware and Dell’s acquisition of EMC. Yes, there was a recent round of layoffs at VMware that was pretty significant. Ultimately, I felt comfortable with the stance VMware is showing and reached out to multiple contacts both inside and outside of VMware to get opinions.

I walked away from those conversations thinking that VMware, though part of the Federation, is very strong in its own right. This company still has new areas to grow and I’d like to be a part of that.

Alright, now that that is behind us, moving on.

Going to the “Dark Side”

I’ve heard this numerous times from my new coworkers and a few others. I understand that becoming a Pre-Sales System Engineer means that I am convincing companies that Product X is right for them, but its so much more than that. At the end of the first week, I questioned why that phrase is even used at all. By the end of the second, lets just say I don’t see it. This is an excellent opportunity and allows me to see and help so many people in so many different environments. I’m here to help validate, demo, show the value of these products and help solve problems. I don’t see a dark side to it at all.

First Impressions on my team and role

I wasn’t exactly sure what to expect going in. This is a vendor role, remember. Up until now, I’ve been a customer. One of the first things that struck me was all of the members willingness to help. People in different areas of the business have gone out of their way to get me slide decks and 1-on-1 meetings to discuss products, helping me get up and running.

My account reps took a good chunk of time to discuss and exchange knowledge and ideas, during a very busy part of their quarter, when they should be focusing on closing deals I obviously am not a part of. Above that, just being available far more than I expect them to be.

But, the best part of this actually happened around going to a customer’s business to do a demo. Now, as I mentioned earlier, I’ve been told “welcome to the Dark Side” plenty of times. This is part of sales, I support sales by doing the technical portion. It’s an understanding that I will convince companies that they need some software to solve their problem or reach their goal. What I wasn’t completely expecting was hearing the account rep and current systems engineer push on the idea of “we don’t sell shelf-ware”. The idea is that we don’t push products you don’t need to solve your current problem, or isn’t part of a soon to be project/goal.

That in itself makes this move all the sweeter. Why? Becuase it goes back to a core competency at my last job, a type of company culture if you will. At Voxeo/Aspect, they called it customer obsession. Doing what is right for the customer and seeing it through to the end. I’m reminded of that by this team and its amazing to see that idea being pushed, even in Pre-Sales.

Drinking from the firehose

Yeah, I have a lot of products to catch up on, but I’ve never been one to NOT want to learn a new enterprise tech. BRING. IT. ON.

Work | Life Balance

Coming from a customer background, I was presently surprised when my phone died the other night and I didn’t have to worry about an on-call rotation. But what really surprises me, is when my manager ends a friday call with “Alright, have a good weekend and remember, Family first… Work Second”. Work/Life balance is incredibly important here and as I agreed to this role, my one concern was how it might affect mine. I had spent a considerable amount of time working on this in my last job to improve it, to the best of my ability. While at VMware, its practically pushed in my favor by management.

Summary:

All in all, I already considered having VMware on my resume as a great career booster, but the perks keep rolling in. I don’t mean that in a way that suggests good discounts or benefits, which coincidentally are also great. The culture is good, the team is great and the role  will definitely cater professional growth beyond what I initially considered. I look forward the future that is at VMware. Now if I could only come up with a paper for VMworld… seriously, what should I talk about?!?!

From Customer to Vendor, I’m making the jump!

While I have been writing posts on professional growth, I’ve also been working on what my next step should be. Like anyone trying to continue their career growth, you take a few things into consideration and draw up a plan.

  • What is the larger end goal for my career?
  • What knowledge and experience am I missing to get there?
  • What short term goals and next steps can I do to help?

It’s with those ideas in mind that I am happy to say I will be joining VMware in March, as a Systems Engineer!

This is a role I felt would greatly challenge me and continue to push me forward into new markets and challenges. It also doesn’t hurt that it’s an excellent team at an already amazing company. I am extremely excited for this opportunity and what it entails for my future.

As I continue forward on this journey, I’ll continue to blog what I see and do. In the meantime… VMware here I come!

This Method Requires Authentication

We were having some issues with one of our VCSA’s and creating or subscribing to Content Libraries. So here is our resolution.

Symptoms:

  • When creating a local Content Library, when clicking finish, it errors with: This Method Requires Authentication
  • When subscribing to another Content Library that has authentication disabled, after copying the json URL into the field and clicking next, it halts the view and states: This Method Requires Authentication
  • When attempting to download Support Bundles from the VAMI at https://<vCenter FQDN>:5480, Downloads timeout and fail

 

SSH into the VCSA and check the following log files:

/storage/log/vmware/vdcs/cls.log
/storage/log/vmware/vdcs/ovf.log
/storage/log/vmware/vdcs/ts.log

In cls.log, you will be looking for something like this:

cls.log
=========
2016-01-20T14:18:01.773Z | DEBUG    | unset-opId       | tomcat-http--39           | SsoOverRestVerifierUtil        | Trying to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/cls/resourcebundle
2016-01-20T14:18:01.800Z | ERROR    | unset-opId       | tomcat-http--39           | SamlTokenImpl                  | Signature validation failed
javax.xml.crypto.dsig.XMLSignatureException: the keyselector did not find a validation key
        at org.jcp.xml.dsig.internal.dom.DOMXMLSignature$DOMSignatureValue.validate(Unknown Source)
        at org.jcp.xml.dsig.internal.dom.DOMXMLSignature.validate(Unknown Source)
        at com.vmware.identity.token.impl.SamlTokenImpl.validateSignature(SamlTokenImpl.java:653)
        at com.vmware.identity.token.impl.SamlTokenImpl.validate(SamlTokenImpl.java:535)
        at com.vmware.vim.sso.client.DefaultTokenFactory.parseToken(DefaultTokenFactory.java:46)
        at com.vmware.vim.sso.http.impl.AuthVerifierImpl.validateSamlToken(AuthVerifierImpl.java:77)
        at com.vmware.vim.sso.http.impl.AuthVerifierImpl.verifyToken(AuthVerifierImpl.java:66)
        at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeaderImpl(SsoOverRestVerifierUtil.java:183)
        at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeader(SsoOverRestVerifierUtil.java:109)
        com.vmware.vcde.common.services.cm.servlet.SsoAuthenticatedFileStreamServlet.doGet(SsoAuthenticatedFileStreamServlet.java:103)
.
.
.
.
2016-01-20T14:18:01.801Z | ERROR    | unset-opId       | tomcat-http--39           | SsoOverRestVerifierUtil        | Failed to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/cls/resourcebundle
2016-01-20T14:18:01.801Z | ERROR    | unset-opId       | tomcat-http--39           | SsoAuthenticatedFileStreamServlet | doGet: SSO verification failed for client <vCenter IP Address>
com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil$SsoAuthException: com.vmware.vim.sso.http.AuthException: The SAML token is invalid!
        at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeaderImpl(SsoOverRestVerifierUtil.java:194)

 

In ovf.log, you are looking for:

ovf.log
-------
2016-01-20T14:18:01.792Z | DEBUG    | unset-opId       | tomcat-http--23           | SsoOverRestVerifierUtil        | Trying to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/ovf/resourcebundle
2016-01-20T14:18:01.804Z | ERROR    | unset-opId       | tomcat-http--23           | SamlTokenImpl                  | Signature validation failed
javax.xml.crypto.dsig.XMLSignatureException: the keyselector did not find a validation key
        at org.jcp.xml.dsig.internal.dom.DOMXMLSignature$DOMSignatureValue.validate(Unknown Source)
        at org.jcp.xml.dsig.internal.dom.DOMXMLSignature.validate(Unknown Source)
        at com.vmware.identity.token.impl.SamlTokenImpl.validateSignature(SamlTokenImpl.java:653)

        
        
2016-01-20T14:18:01.805Z | ERROR    | unset-opId       | tomcat-http--23           | SsoOverRestVerifierUtil        | Failed to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/ovf/resourcebundle
2016-01-20T14:18:01.805Z | ERROR    | unset-opId       | tomcat-http--23           | SsoAuthenticatedFileStreamServlet | doGet: SSO verification failed for client <vCenter IP Address>
com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil$SsoAuthException: com.vmware.vim.sso.http.AuthException: The SAML token is invalid!
        at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeaderImpl(SsoOverRestVerifierUtil.java:194)
        at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeader(SsoOverRestVerifierUtil.java:109)
        at com.vmware.vcde.common.services.cm.servlet.SsoAuthenticatedFileStreamServlet.doGet(SsoAuthenticatedFileStreamServlet.java:103)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
        at com.vmware.vcde.common.services.cm.servlet.DispatcherServlet.service(DispatcherServlet.java:53)

 

In ts.log, you are looking for:

Ts.log
---------
2016-01-20T14:18:01.792Z | DEBUG    | unset-opId       | tomcat-http--14           | SsoAuthenticatedFileStreamServlet | doGet: Entering (/ts/resourcebundle)
2016-01-20T14:18:01.805Z | ERROR    | unset-opId       | tomcat-http--14           | SamlTokenImpl                  | Signature validation failed
javax.xml.crypto.dsig.XMLSignatureException: the keyselector did not find a validation key
        at org.jcp.xml.dsig.internal.dom.DOMXMLSignature$DOMSignatureValue.validate(Unknown Source)
        at org.jcp.xml.dsig.internal.dom.DOMXMLSignature.validate(Unknown Source)
        at com.vmware.identity.token.impl.SamlTokenImpl.validateSignature(SamlTokenImpl.java:653)

2016-01-20T14:18:01.805Z | ERROR    | unset-opId       | tomcat-http--14           | SsoOverRestVerifierUtil        | Failed to verify request signature using following; host:<vCenter FQDN>, port: 443, uri:/ts/resourcebundle
2016-01-20T14:18:01.806Z | ERROR    | unset-opId       | tomcat-http--14           | SsoAuthenticatedFileStreamServlet | doGet: SSO verification failed for client <vCenter IP Address>
com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil$SsoAuthException: com.vmware.vim.sso.http.AuthException: The SAML token is invalid!
        at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeaderImpl(SsoOverRestVerifierUtil.java:194)
        at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeader(SsoOverRestVerifierUtil.java:109)
        at com.vmware.vcde.common.services.cm.servlet.SsoAuthenticatedFileStreamServlet.doGet(SsoAuthenticatedFileStreamServlet.java:103)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)

 

Cause:

According to VMware support, these log entries show no security context for the user. Without that Security content the user cannot perform actions on the content library.

 

Resolution:

We found the signing cert and its root CA used by SSO from vmware-identity-sts.log and took out the ssoserverSign and the root certificate and added them to the CA to TRUSTED_ROOTS using the below mentioned vets command.

/usr/lib/vmware-vmafd/bin/vecs-cli entry create --store TRUSTED_ROOTS --alias roo51 --cert 51root.crt
/usr/lib/vmware-vmafd/bin/vecs-cli entry create --store TRUSTED_ROOTS --alias roo52 --cert 52root.crt

 

Then restart all services. Run the following commands. (No there is not a –restart or –reset, use both commands).

service-control --stop --all
service-control --start --all

Thats all for today folks. Hope this helped!

part2-featured

Professional Growth: Part 2 & 3 – Base Requirements and Upgrades

Continuing the series on professional growth, which started with my last post: Part 1 – Inspirational Beginnings. I started off asking “How did you get started and how long have you been in the game?” I got some interesting results and this next post I asked the question: “What traits, methods, knowledge or experiences did you find crucial to your growth?” Also updated with question 3, “What has changed since then, what’s new to be done?” Keep Reading

part1-featured

Professional Growth: Part 1 – Inspirational Beginnings

I asked a number of tech professionals in varying lengths of carrer, heights of their career (Admins through Upper Management) and areas within the technology industry (Programming, Marketing, Infrastructure, etc…). I did this because I understand that getting started is hard. Either getting out of school or finding opportunities while in helpdesk. So I am sharing this to provide insight for those looking to grow professionally both inside and outside work. Keep Reading

Answering My Own Questions

This week I reached out to a bunch of people in my followers list with a small list of questions. The purpose of this is to gauge what people in different roles, years of experience and geographies had to say about growth in the technology field. I’ve received some of the responses already and I have to say, I’m loving them. I can’t wait to do a write up. Just forgive me for the length, because there will be SO MUCH that I want to share from them. Keep Reading