My most epic project

The biggest project of my career was a Y2k project for a state agency. The project was a huge challenge by itself. To make things more interesting, it had a fixed timeline (Y2k) and a customer who didn’t typically do well with fixed timelines. That combination is a near guarantee to make things challenging at times. It was a pretty big opportunity for me and I was optimistic about it, in spite of the obvious challenges.

On day 1, we started by formulating our strategy. We needed a set of ground-rules that would guide our decisions, to ensure success. This was the crux of our strategy:

  1. Contain the scope (and be vigilant about it)
  2. Build a solid proof-of-concept (program/prototype)
  3. Focus on architectural models that would prove each of the elements of the final product (eliminating unknowns and accelerating development)
  4. Train the staff
  5. Cooperate
  6. Commit to improvement (within reason)

We figured that if we were committed to those guidelines, then our development would accelerate (meaning: the rate of completing work would steadily increase), the timeline would not be able to outrun us, and the chance for surprises would be minimized. The timeline would be predictable. We could do it, on-time.

We started-off strong with all six of these. Of course, progress always seems to start-off a little slow during the first few weeks of any project. It is a real gut-check until you pick up steam. I was only 30 years old, and I got my first 5 gray hairs during those 3 months.

During that time, my dev lead expressed concern about making the timeline. The task was gigantic and he couldn’t imagine how we could accomplish so much in such a short time. I explained how it would all work (a few times) but he still had concerns. He had never been part of anything like this (and technically, neither had I). He wasn’t as optimistic as me. In fact, you could say he even sounded somewhat pessimistic. I figured, he probably just needed some solid reassurance from me.

After trying a few times, to reassure him, I realized that there wasn’t anything that can be said to reassure a young developer, about a project as big as ours. So, I changed my pitch: “If you honestly don’t think that we can complete this, then we really only have two options: 1) Find new jobs right away. Or 2) Work as hard as we can and do our best, and at the end of it all, if we come up short, we can rest-assured that nobody else could have done a better job than us and been more successful”. (of course, I was being a little sarcastic with the first option).

He acknowledged that there was really only one reasonable choice: do our best, and leave nothing on the table. So he committed to our endeavor. I did point-out that I was personally determined to do anything necessary (within the boundaries of good ethics) to get the project done on time. We had some great support and resources at our disposal. We were even told that “getting a B” would still pass, but we were going to settle for nothing-less-than an A.

We pressed-on with diligence, constantly checking our quality, progress and vigilantly containing scope. One time, our scope-containment effort even resulted in a rather heated disagreement with a serious player. I was a small dog in a fight, but I treated it like the careers of my team (18 of us) depended on this decision. We stood our ground and worked our resources pretty hard. I used up all of my “get out of jail free” cards that day, but prevailed. The scope was contained and we sprinted towards our due date.

In the end, we finished 2 weeks early (Sep 16, 1999) and the program exceeded the expectations of everyone, for performance and stability. Everyone was astounded. We rolled-out the project to the entire state of Michigan, over the next few months. 14000 state workers picked it up without a hitch. Y2k was a non-event for my team.

Of course, I cannot take sole credit for the win. I had a great PM, excellent dev managers, I had one or two certified geniuses on the team and management was behind us 100% (even covered my backside from that dog fight). We couldn’t have done it without all of them. Each accomplished some legendary feats during the project.

I still feel that the winning ingredient in all of it was the solid foundation (6 guidelines) and the commitment from everyone. At times, we got scared and wavered, but that fear also drove us to accomplish things that (I’ll even admit) were pretty colossal. We were driven and that made all of the difference.

Since that project, I have not faced very many challenges that can compare to that one. I am fortunate to have seen how a great accomplishment can be achieved. I was there and I know what goes into it.

If you ever get offered a chance to accomplish something epic, do a gut-check, be realistic about your own resources. If you go all-in, be ready to back it, all the way. It might be the most legendary win of your life, but only if you apply the effort needed to get that win.

Posted in Career, Lessons Learned, Methodology | Tagged , , , , , , | Leave a comment

Event code: 3008. A configuration error has occurred

This week, we were deploying some new files to our project (running on IIS). After adding the files, the app didn’t run, and we got the following error in the IIS logs:

Event code: 3008
Event message: A configuration error has occurred.

… [snip]

Exception information:
Exception type: ConfigurationErrorsException
Exception message: Could not load file or assembly 'ProjectName_old' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
at System.Web.Configuration.CompilationSection.LoadAssemblyHelper(String assemblyName, Boolean starDirective)
at System.Web.Configuration.CompilationSection.LoadAllAssembliesFromAppDomainBinDirectory()
at System.Web.Configuration.CompilationSection.LoadAssembly(AssemblyInfo ai)

The error looked like maybe there was some kind of config error. Except we didn’t make any changes to the config. We just deployed a new version of a DLL in the /bin folder.

Something about it sounded familiar. I’ve seen this misleading error a dozen times before.

The problem was caused by a simple mistake on the part of the server admin. He renamed old dll before he put the new one on the server. Unfortunately, the way IIS works, it will read all DLL files in the /bin folder and pull them into memory. Unfortunately, this will cause some serious conflicts if you have two copies of the same DLL, but with different names. IIS won’t know how to handle the conflict.

It would be great if the error message said something accurate like “DLL Conflict: two DLLs contain the same classes/namespaces and are conflicting with each other. The files are named Copy1.dll and Copy2.dll” but nooooooo. Instead it gives this dandy error message.

So, hopefully you know better now. If you ever deploy content to an IIS server, please just zip the whole folder and store that backup somewhere (not in the wwwroot, etc). Then delete the contents of the folder, and then put the new content there. It is the safest way to avoid conflicts, while preserving your options to restore everything, in the event of a problem.

Posted in Errors, IT Horror Stories, Lessons Learned | Tagged | Leave a comment

Underachieving, overachieving

When I was in high-school, I had an interesting discussion with a guidance counselor. I was taking some pretty hard classes in school and I was getting so-so grades (B or C, not A). His advice was to take easier classes. I told him that I really liked to be challenged and I don’t think I would get better grades if I had easier classes. I confessed that the problem was that I was just lazy.

He persisted that my grades were going to be used as more of an indication of my intellect, and the challenge/difficulty-level of my classes would be overlooked. Basically, if I wanted to go to college, I needed good grades, not a good education. He told me that my grades made me look like an under-achiever and that was really bad.

After high school, I went into the US Army for a while and did okay. Some things, I did great but others, not so much. A few people gave me great advice about self-improvement, but I had trouble embracing it. I must confess that I was still a little lazy and I moved a little faster when a sergeant or officer was um, “motivating” me.

In college, things turned around. My junior year at U-Mich, my classes got pretty hard and the dean of engineering told me I had to bring up my grades or they would kick me out. I couldn’t believe it. What a wake-up call. I cancelled a lot of my leisure activities and replaced that time with studying. I even started studying things that I didn’t have to study. I started learning some really cool stuff and wanted to apply it.

After graduating, I kept applying this idea: I looked for things to do or learn, which went beyond what was asked of me. At work, I started looking for new ideas for learning and growth. I started looking for the things that needed to get done, but weren’t, or places where I could apply some newly-gained knowledge, to make something work-better or simplify something. It was pretty fun and satisfying.

Unfortunately, I got a little carried away and started paying too much attention to these other great/fun/rewarding things and not enough to the work that was assigned to me. My boss had to have a talk with me and set me straight.

It is funny about how these things come back around. I have also seen this among some of my co-workers. Some work is just so much fun to do, and some of it is boring and tedious. At work, you might begin to do the fun stuff and let the boring stuff slip a little. Maybe too much.

If you are thinking “this doesn’t happen to me”, then answer this question: What have you done for paperwork this month? Written any docs lately? How about your TFS check-in comments? I’m not trying to name names, because I’m certain we all do this sometimes.

My point is this: at work, you still get a grade. It just isn’t on a report card. It is good to be an overachiever, but don’t lose sight of how it affects your grade. Do your homework, study for the test, be thorough. Go for that “A”.

Posted in Career, Education, Lessons Learned, Professionalism | Tagged | Leave a comment

EMV transactions in the iSC480

My boss’s boss was informed that the credit card laws were changing and therefore, we needed to deploy new credit card readers and change our program to interface with the new readers.

I had already begun developing and was interfacing with the iSC480 very well (see previous article for more details).  My one remaining task was to detect when a user was trying to “swipe” an EMV card.  The EMV card must be inserted and not swiped.

The documentation said several times, “If the swiped card is an EMV card then the cardholder will be prompted to insert the card” (Dev Guide, sec 3_6_18, variable 413).  Well, that never happened.  So I had to write that code myself.

It was pretty easy.  I just had to determine if a card is an EMV card.  Track 2 contains a service code (positions 21-24).  If position 21 contains a “2” or “6”, then the card is an EMV card and must be inserted.  I detected it and changed the screen message and I was back on track again.  No biggie.

When I inserted a EMV card, nothing happened.  The vendor said that the device should start automatically running through screens.  I never saw any of that.  Finally, my team had a phone conference with a tech guy from the vendor.  He pointed out a few VERY important pieces that were necessary to process EMV transactions:

1. EMV must be turned on in the device.  By default, it is NOT turned on.  So, you will never get anywhere until you know this critical step.  To turn on EMV in the device, you need to send a “M60” command to change setting “19” to a “1” instead of the default of “0”.  The support guy was surprised that I hadn’t read that tidbit because it was “easy to find” somewhere in the 826-page-long PDF document (Dev Guide).  I eventually found it under the section titled “Support for Voice Referral for EMV”.  I guess they figured the whole “voice referral” part would not dissuade me.  They were mistaken.

2. Once the user inserts the card, a 09 command is received (via the API). At this point, you (the programmer) needs to send the following:
– M14 (use transaction type 01 = Sale)
– M13 (Set amount, without the decimal point)
– The device will respond with a 33.02 message
– M04 (set the transaction to “B” for credit or “A” for debit, etc)
– M13 (set the amount again, for some reason)

Then, the device will take-off and do all of the EMV magic.  The user will have to do some actions on the screen.  These actions will vary, based on the settings inside of the EMV chip in the card.

3. When the user completes all of the required input, the device will send a 33.03 message asking you (the programmer/program) to do a credit auth.  You need to respond with a M22.04 message.

4. The device will ask the user to eject the card.  The device might ask the user to sign.

5. When the user is done signing, the device will not give any indication that the user is done signing.  That is of course, unless you already knew that you had to set variable “0009” to “1” and you have already done it, before you started any of this.   If you already took care of this, the device will send a “20.0” message to indicate that the signature is done. Otherwise, you could always set up a polling loop (timer thread) to check if a signature message has been recently received.

6. Done.

So, as long as you know all of these little tricks, the EMV stuff is pretty easy to deal-with in the iSC480.  Otherwise, the EMV stuff may not seem very intuitive and it will have you scratching your head for a while.
I was disappointed that, Goog/Bing didn’t have any info about it. I guess people aren’t supposed to talk about the SDK. I have been intentionally vague in this article, to avoid heat. So, if this article didn’t make any sense, it is probably because you don’t already have the SDK.

Final note: Snaps to the tech support at the vendor. I always hate calling for tech support. It feels the same as stopping at a gas station and asking for directions. *ugh*. However, in this case I wish I hadn’t burned two weeks before I called. The support folks at the vendor, got me back-on-track quickly. The phone call and solution took like, 5 minutes.

Hopefully, you found this guide on the web and it saved you a lot of headaches.

Posted in Programming | Tagged | Leave a comment

Training your manager

Disclaimer: This is not about my current workplace. This is about all workplaces.

Whether you are a team lead, software architect, PM or a junior developer, you count on your team for stuff. Maybe if you are a one-man-team, you do this to a lesser-degree, but you still would have to count on yourself to fill those roles.

Each of the roles on a team, are there because they have a function in the process of writing software. Everybody has a job to do. If every job is done well, the whole thing works like a charm.

Believe it or not, management also has a job to do. I’ve met people who honestly thought the job of management was simply to watch you work and badger you. I’ve known others who thought the job of management was to set up snares to catch you when you made a mistake and then make you pay for it. I’m inclined to think that those people just didn’t have a realistic understanding of things. Maybe they were pessimistic or obtuse, or maybe just downright paranoid.

The really good managers are always there to help and you can see the value they contribute. For example, a good manager will do the following:

  • Ask if you need anything to do your job. If you need something, they acquire it for you or help you acquire it. (within reason)
  • Attend meetings for you and handle issues, so you can focus on your work.
  • They keep you informed, so you understand the big picture and see how you are contributing to the success of your organization.
  • They plan ahead. Short term plans are very detailed. Long term plans are more vague, but become more detailed as they approach.
  • They work with you to determine solutions and estimates, and technologies.
  • They do more asking than telling.
  • They encourage you to think about how you can grow your organization and yourself.
  • They get you the recognition you deserve.

Of course, not all managers are good managers. The career goal of a manager is to make other management believe he is doing a good job. That is how you get advancement. However, this is easily confused with the higher goal, of making management believe his people are doing a good job. Note the difference “I am doing a good job (regardless of my people)” vs “My people are doing a good job (therefore, I must be doing a good job)”.

Not all managers are born with the innate ability to recognize this higher goal, but that is no reason to distress. If you find yourself in the hands of a manager who is not taking care of you, you just need to work on that person, like you work on any of your other skills. Managers can be trained, they just need to see the carrot.

If your manager has some room-for-improvement, try asking him do to the stuff that you need him to do. Help him understand how this helps you and moreover, helps him.

  • If he is not giving you recognition, try showing him how this is done. Write a weekly document that highlights a few things
    • Accomplishments for the week
    • Plans for next week
    • Roadblocks or resources that you need
  • If you deliver something really valuable (like a weekly document highlighting accomplishments) and the manager says you don’t need to do this. Tell him that you like to do this and you know of other managers who ask for this kind of thing. Therefore, you feel obliged to do this yourself. Then mention how it must make his job easier, because his manager must need this kind of info from him. He can just copy it out of your status report and send it upward.
  • Do your own monthly evaluations and invite your manager to participate. After a few of them, the manager will see the pattern and join-in.
  • Find out what other kinds of paperwork a manager should be doing. Do it for him and give it to him. He will see that it is not that hard (since you were able to do it). Also, modifying an existing document is much easier than starting from scratch.
  • Make the time to talk about progress. Don’t just talk about the current problems, talk about small solutions to the current problems. Build momentum and a “can do” attitude. Turn it into a habit.

If you’ve never done this for a manager, you might be thinking that I must be kidding. It sounds like I’m suggesting that you do your manager’s work and he will get all of the credit for it. Well, you are half right.

This kind of paperwork does of course benefit you too. Visibility is rarely a bad thing for you (unless you have something to hide). Plus, some day, you might take a vacation day/week. That is a great time to pass the baton.

Training a manager might be something you’ve never tried, but like anything, you can learn to do it well, and reap the benefits.

Posted in Career, IT Psychology, Lessons Learned, Professionalism, Team | Tagged , , | Leave a comment

Magnitudes of technical debt

The term “technical debt” really covers a lot of areas of concern. It is not simply relegated to “messy programming”. That is part of it, but there is more to it.

Like with many unpleasant things, the impact of technical debt, can be measured in magnitudes. Some kinds of technical debt are easy to deal-with and some are monumental. When a developer says “let’s resolve the technical debt around here”, be careful about writing a blank check.

Like with all development efforts, it is a good idea to “contain your scope”. You will generally maximize ROI and build momentum when you start with the low-hanging fruit and avoid items of high impact.

Low impact tech-debt (quick ROI) examples:

  • Compiler build warnings – These are usually easy to resolve. Make a habit of resolving these as they appear, or work to eliminate a dozen of these a day/week.
  • Coding conventions violations – When your code is “conventional”, it is easier to maintain. Pick a coding convention and enforce it. Then there won’t be “my code” and “your code”. It will all look identical.
  • Configs – Hard coded settings in a project can be evil. You don’t want to modify your program every time you move it to a different server or file path. That is bananas, and very risky. Make sure you have those settings in config files. Make sure the configs are clean.

Med impact tech-debt examples:

  • Obsolescence, falling behind – Don’t let your program turn into “the next COBOL”. Even if it is actually written COBOL, you can still take periodic steps to keep it updated and modern.
  • Awkward programming styles – Ninja programmers can come up with some impressive stuff, but you don’t want your code written in a manner where only a genius can maintain it. If you haven’t generously applied “the kiss principal”, then you are carrying some tech debt with a heavy burden, and it will cost you.
  • Lack of error handling – If a user complains about your buggy program and you have to ask what they did when an error occurs, then you don’t have adequate error handling. Your error handlers are metal detectors to show you the needles in the haystacks. You don’t ever want to live without them. Be generous with error handlers and logging.
  • Inadequate tracing – See error handlers, and imagine that something went bad, but didn’t result in an error that could be caught. Without a trail of breadcrumbs, it will take a frustrating amount of time to recreate the conditions. Eventually, you will wish you had tracing to tell what was happening.
  • Things like: global variables (a needle that seems to move around in your haystack)
  • DLL hell, GAC hell, SP hell, all of the other hells that come from reuse/sharing of code. – Brochureware always sells the idea of code reuse, like it is so great and never turns your life into a nightmare. If you’ve been around the block, you know better. Caveat emptor.

High impact tech-debt examples:

  • Architecture – If your architecture paints you into a corner, you might be in the worst kind of checkmate. Starting-over, might be your only hope.
  • Bad designs – There are all kinds of ways to design a system, which makes them impossible to maintain or close-enough to it. Untying some knots, can turn into epic tales.
  • Security – Thanks to the internet, security is a big enough topic that it a career path for many people. Dabbling in security is like being your own lawyer. It is not a good place to learn from your mistakes.
  • Platform (OS, data, language) – I’ve always laughed at IT people who warned you about a technology like SQL Server or .NET because “you don’t want to be locked into a single platform”. It is a ridiculous claim and the people who fall for it eventually realize that people don’t switch platforms for any good reason. If you ever do switch platforms, you will never do it twice. Locking into a platform is a way to get stability. Y’know, like a foundation. You don’t just switch it periodically.

I’m not saying you can never tackle the topics on the High list. Sometimes you just need to do those. However, don’t do it on a whim. Rome wasn’t built in a day. Don’t start with Mt Everest.

Most important of all: The time to be the most careful with these things is BEFORE you start programming and not afterwards. Like a foundation, you want to start with a solid one, in a reliable place. Make sure the big items are done correctly by someone who knows their stuff (and not just some silver-tongued sales person or giddy fan-boy).

In the words of Bertrand Russell, “Fools and fanatics are always so certain of themselves, but wiser people so full of doubts.” Measure yourself by those words.

Posted in Architecture, Lessons Learned, Review | Tagged , , | Leave a comment

LLT – M53: unknown product code (add it in ‘LLT.prm’ file)

Once again with the LLT. I just got a new PC and had to reinstall all of my software and re-learn the tricks for connecting LLT, etc.  Btw, the help file for LLT has better directions than the rest of the docs. I wasted an hour trying to figure out how to get the device into LLT mode.  Hint: read the LLT docs, section 6.  Btw. the “up” key is the minus -.

When I tried to connect LLT to my device, I was getting this connection error:

“M53: unknown product code (add it in ‘LLT.prm’ file)”

The LLT.prm file is in the same folder as your LLT.exe.  It has a list of all of the possible devices, but it stops at M52.  The iSC480 is supposed to be M53.  For some reason, in the past 2 years, nobody has managed to update this file.  *ehem*.  So we have to do it ourselves.  No biggie, I guess.

Use notepad.exe to edit the LLT.prm file.  Put “M53: iSC480” on the last line (before the line that says “M??:all products”).  Save, and re-launch LLT.  Since it probably took you five minutes to make the change, your iSC480 has probably restarted, so you need to do that trick to get back into LLT mode. 🙂

Btw, the entry for M53 is optional.  The LLT will connect to your device and work etc. even with this error.  It is really just a warning.

Posted in Errors, Lessons Learned | Tagged , | Leave a comment