Technical Bankruptcy

*** Disclaimer: This is not about my current project, it is about every project everywhere ***

Technical-debt is a natural byproduct of software development.  If you don’t recall seeing technical debt in every sw-dev project, it is because most developers prefer to resolve technical debt immediately as it appears.  However, if you have a project with an aggressive timeline, you will see technical debt and you will probably accumulate some of it. This is a natural byproduct of aggressive timelines.  The more aggressive the timeline, the worse the byproduct.

The reason technical debt accumulates is fairly obvious.  It takes time to resolve/remove technical debt.  When developers are pressed-for-time, they have less time to review and refactor their own code. They have to decide between “clean it now” or “put-it-off till later”.  When the program works and it passes testing, only the developer will know the difference.

Too Much Debt

If you’ve never heard of “Technical bankruptcy”, write-it-down.  It is a thing.  You know you have reached the point of Technical Bankruptcy when you realize that you are overcome by an insurmountable amount of technical debt.  You no-longer have the time and resources to resolve the tech debt because you are slowly suffocating under a creeping blanket of support calls/tickets.

If you are not sure if you have reached technical bankruptcy or not, take this quick quiz: Is anyone on your team saying “We would be better-off if we just started over from scratch. Let’s throw it all away and start over”.  When you believe it is better to “start over”, you have officially reached the point of technical bankruptcy.

We’ve all seen it before.  It is pretty bleak.  It is hard to imagine how you got to this place. “How did things get this bad?!”  It is even harder to acknowledge the real cause of this, because everyone ignored the problem and in-some-way allowed, caused or contributed to it.

Turning things around

To avoid technical bankruptcy, you need to actually pay-down your tech debt.  Unfortunately the very-nature of technical debt is that it slows you down and prevents you from developing software effectively.  If you are spending all of your time, dealing with complaints, you have very few options.  If you haven’t already done so, you absolutely must cut-back on other projects and assign a higher priority to the process of paying-down-debt. If I’m making it sound easy, let me correct that notion.  This is going to require a significant effort.

On the other hand, if you think you will do better by starting over, think again.  Before you try it, you really need to take a deep look at yourself and your processes, so you don’t arrive at the same point, over-and-0ver again.

It is a long road back and it begins with things like “living within your means”, being realistic about your ability to produce and maintain software.  Committing to improving your processes and paying attention to quality. This last concept is the most significant one.

It is a lot of work and a lot of commitment, but you can do it. Keep your head above water.  Live within your technical means.  Don’t accumulate technical debt.  Stick with these rules and some day, you won’t have to spend all of your resources, paying interest. Remember: Accumulating debt and never paying it down, will lead to one place: bankruptcy.  Don’t go there.

In my next series of posts, I will be discussing technical debt, why your manager loves it, and some things you can do to monitor and manage it.  So check back next week.

Posted in Lessons Learned, Review | Tagged , , , | Leave a comment

Current Mission Statement

When I read the book “7 habits of highly-effective people”, I remember my reaction to the suggestion about having a “personal mission statement”.  It struck me as being a little silly.  Am I really going to do that with all aspects of my life?  Do I really need that sort of thing? Where would I even start?

Well, at work my attitude is much more focused.  At home I can wing-it, whenever I want, because I am still figuring some things out and sometimes it is more fun to live with fewer boundaries.  Honestly, I spend a lot of my time at home pondering problems from work instead of working-out problems at home. So I’m more inclined to have a plan for work.  Regardless, I apply much more structure to my work goals, regimen and mission statement.

I just thought it would be interesting to some of you to see my current mission statement.

  1. Provide the best value that I can to my employer and customer
  2. Determine what-it-is that the customer wants
  3. Determine what is out-of-scope and counter-productive, regardless of its value
  4. Never allow my ego to get in the way of the customer’s needs
  5. Watch my zones
  6. Determine communication routes and routines
  7. Plan ahead as much as possible, but adjust the plan, as needed
  8. Always have several backup plans
  9. Be transparent
  10. Identify the (increasing) points that the bar can/should be raised-to
  11. Determine the path to accomplish this (increasing improvements)
  12. Be patient and try not to agitate my colleagues, while pursuing these goal
  13. Always do my research before opening my mouth
  14. Prepare proof-of-concepts before discussing ideas for fixes
  15. If something seems strange, find out why it is that way, before proposing to correct it

There are a few more things that I do, but most of my decisions are founded on these items, and I do my best to weigh my choices against this list.

Every month, I review my mission statement and make sure my plans and work are in alignment with it.

Posted in Professionalism | Tagged , , | Leave a comment

Just like free money, for SQL server

I just discovered something today which got me very excited, like finding a $20 bill, laying on the ground.  It must be my lucky day!

Each time a new (or upgraded) development tool or server product comes out, I like to read-up on the new features. For example, when C# 5.0 came out, I read about the new features like .? which is very cool and solves OO problems that I’ve had to overcome my entire career.  Knowing about the new features is great because those things usually save you a lot of time or headaches.  You just have to know they exist and how to use them.

Today, I stumbled upon a feature in SQL server called “Data Compression”. Imagine this: if you could pick some (maybe not all) tables in your SQL database and have them compressed (eg. Zipped).  It would save space, right?  That alone would be really great.  My mind is already racing with a list of tables that I would apply this to.  Then I started thinking, “There must be a trade-off.  It would slow things down a little, while it zipped/unzipped the data”.  Therefore, I would have to apply it judiciously.

I’m glad I read more about it, because I nearly overlooked the bigger picture. If you have ever done performance studies on databases (DB servers), you will quickly find out that most databases are not CPU constrained.  They are hard drive constrained.  No matter how fast your SAN is, it is never enough.  So, imagine enabling a feature in your database which could increase your disk throughput by a factor of 8!  Aye-carumba!  Right?  Yeah, so it turns out that when you compress your SQL tables, many will compress to (approx.) 1/10th their size (YMMV, depending on the content).  It also stays compressed during reads and writes.  Which means you are moving compressed data from disk to memory, and only de-compressing it in-memory.   Of course, if you have a query which does a table-scan, it would be noticeably slower, but if it is indexed, the index scan is still really fast.  You can even compress your indexes (for storing really big indexes which don’t change often). Indexes also are compressed on disk and decompressed in memory (only).

The concept of 1) using less disk, 2) using less disk bandwidth and 3) applying it at a per-table level, is just extremely compelling. I was like “Heck yes, I MUST upgrade to SQL 2016 immediately!”  Then I discovered that this feature has been there since SQL 2008.  So, if you have been hiding under a rock for the past 9 years, like I have, it is time to climb out, take a fresh breath of air, and pick up all of that free money laying on the ground in front of you.  Cha-ching!

To learn more, check out this great article. It does a magnificent job of explaining this free money and the various ways to apply it and the various tradeoffs:

Posted in Optimization | Tagged , , | Leave a comment

Architectural review phrase book

When a project is in its early stages, (or sometimes, much later), it is time to have your system designs and software/system architecture reviewed. When this happens, it is not unusual to have a difference of opinion with your reviewer(s).  In fact, I think it is very healthy to talk-about and consider alternatives, and then weigh the merits of those alternatives.  This indicates that everyone is thinking and keeping open minds.  It can produce some great results.

Usually, this is a good thing, especially when you are dealing with reviewers who are professional, experienced and knowledgeable. I say “usually”.  Unfortunately, you may land in the hands of someone who is not those things.  In that case, you may find yourself doing more arguing than reviewing.  Hopefully, you will still be able to reach some level of consensus.  However, that is not always possible.

Those situations are frustrating and even painful, but in retrospect, some of the worst reviews can be remembered as being the most entertaining, someday.

So for your entertainment, here are a few phrases, taken from bad reviews, which may sound impressive but are total BS. Anyone who utters these during a review, is just filling the air, and nothing more:

  • This is what everyone does
  • This is the right way
  • I have seen this done in every project
  • Nobody does it that way any more
  • I can’t name a single person who thinks this is a good/bad technique
  • That approach is a joke
  • (insert your own clever insult here)
  • (insert other grandiose statements that imply your idea is super-great, or someone else’s idea is stupid or amateurish)
  • That technique is called [pick some un-friendly sounding term, like “hard coding”]. It is bad. You should never do that.

You might recognize the pattern: It is all conjecture and opinion. Claiming that something is a pattern, or something is stupid, does not say very much.  You will easily spot which person is the hack, because he/she will not be able to actually explain why an idea is good or bad (other-than, claiming that he/she knows more than you do).  Bickering, or flamboyance, is not legitimate proof.

In contrast, here are some phrases that will actually get something done:

  • Please tell me about some alternatives and what they give me over this design
  • Have you used this design on other projects? How did that turn out for those projects? Any problems or concerns?
  • What was the size of the project (where you last saw this done)?
  • Let’s discuss your alternative to this, and discuss what it gives us, in-contrast-to this design
  • Okay, so there are some clear benefits to using another design, but are those benefits as relevant here for this project?
  • Will that fit into our timeline, without adding people or money?
  • Can you show me a model of this design? I would like to see how it works

You can quickly see that these are phrases that an open-minded person would use. Which is not to say that all alternatives are good ones.  It is really easy to spout about the greatness of some technology that you’ve never used.  It is a whole different thing to say that you’ve personally used a technology on a project with many similarities. Basically, it worked then, and we are still talking “apples vs apples”.

One more “BS meter” test: there are very few things which are good all of the time, or bad all of the time. When something sounds too rosy, I need to hear about the down-sides.  If the proponent cannot produce some negatives, then I would not trust the endorsement. Even GOTO statements have a legitimate place in the programming world.  Do your own research, until you find the real truth.

Keeping Your Eyes Wide Open

People who talk like the first set of statements are really trying to impress you with big talk, in the hopes that you will feel embarrassed and simply yield. It is a weird and dangerous ego fluff. If you succumb to it, you will unfortunately be the one who is holding the bag.  In contrast, the big talker will move-on and offer dodgy advice to many other people, with no repercussions.

Unfortunately, I have seen dozens of projects that went down terrible and destructive paths because of stuff like this. If you hear talk like this, you really should cut it short and find a better reviewer. The longer the other person spouts hot air, the more exhausted you will get and the more you may feel like succumbing to the BS, just to find the nearest exit.  If you feel that way, just put a halt to it, and walk away.

Not all elitism smells the same. When you encounter some, please apply judicious scrutiny.

Posted in Lessons Learned, Professionalism, Review | Tagged | Leave a comment

Scaling a project

I recently read an article from a local “expert” about scalability of a dev team. The author claimed that every project can only be completed by a team of 7-10 people. The size of the project didn’t matter to the author. The limit is 10 people. No exceptions.

As I read it, I thought maybe he was trying to be ironic or maybe he would do a reversal for theatrical effect.  No. He simply tried to build a case for why he was correct by listing some large projects which had failed.  I guess he was not aware of any successful dev teams who are larger than 10 people.

As I thought further about his statement, I recalled a conversation that I had with a manager, earlier this year. He asked me how many people I could manage. I started with a similar answer; I could directly manage 7-10 people, but if we needed to scale beyond that, we would need to change our team structure and project structure. Just because I can’t personally manage more people, does not mean that our project needed to be limited by me.

Science of Scalability

During my senior year at U Mich, I took a class called “distributed processing”. It is a fascinating topic. I enjoyed it so much that I have studied it my whole career with hopes of applying it as often as possible. As it turns-out, the concepts are applicable to people, and other things beyond just computer processing.


In distributed processing, a huge hurdle to overcome, is the fact that distributed processing is not practical for all situations. To effectively know whether distributed processing, is the right answer, you must first assess the following:

  1. A task must be divided into sub-tasks that can be distributed. Some tasks easily lend themselves to this concept and some are impossible (or close). Determine the effort to sub-divide your task. Add that to your overall processing time/cost.
  2. After the distributed tasks have been processed, the results need to be merged. Determine the effort to collect and merge the results. Add that to your overall processing time/cost.
  3. If some tasks take longer or shorter, you need to wait for them all to complete, before you can continue. Therefore, you will never be faster than your slowest task.

Once you sum these factors, you can see that it is possible to take longer to [split-up a task, send it somewhere else, get the results and merge it], than it would have taken if you just did it locally.

Therein lies the challenge for distributed processing. Finding processes that can be distributed effectively.  The easy answer is “Don’t bother. It isn’t worth it”, but that is not always true.

Sometimes, there is more than one approach for each of these steps. If you are aware of each different approach, you can evaluate which one will give you optimal performance for your situation. YMMV.  This reveals the bigger challenge.  Your first guess might be wrong, but there might also be a better answer and you haven’t found it yet.

Distributed Processing for Projects

The US Army applies this philosophy every day. Without this, an army could only scale beyond “one big bunch of fellas”. You can imagine how ineffective this would be. Clearly when you need a bigger team, you need to determine a way to scale-out your people.

The key to keeping this all working effectively is organizing them. Everyone knows that this is the value of good “management”.

I suppose some people don’t get it, because they focus on doing their own job well. They have no idea what other people do, unless they do something for you (like HR). Most people rarely (or never) ask a manager to do something for them. So their only interaction with a manager is frequently receiving more work and being asked to do more work, faster. If you have never taken the time to look beyond your work-load, you may never understand why your boss does this. In fact, you will probably be frequently irritated by your boss and might even become spiteful.

Applying it to Your Project

The solution is to realize that management is “a system”. You can learn how these things work and why. You can see how different ones work and what makes them more effective (or less effective) under different circumstances. This is the same thing a SW architect is supposed to do; understand systems, components and options. Determining how to scale a project is like learning how to scale a system.  Learn the parts, know how/when to apply them.  Know how to measure their effectiveness.  Apply, measure, adjust.  The concept is the same.

When you learn this, you will see that management is just another component that you can add into your project, or change as needed. So, to scale your project, a) acquire the necessary skills, or b) get someone who is already good at it.
Don’t be a bottle-neck.  Be ready to overcome limitations. Grow (personally), and scale yourself and your project as needed.

Posted in Lessons Learned, Methodology, Optimization, Team | Tagged , , | Leave a comment

Grave naming

I just heard a new one today.  On the topic of “software development and coding standards” have you ever heard of “Grave Naming”?

I guess it refers to the practice of standardizing specific domain (value) names across development layers.  (…from the cradle to the grave)

For example, if you have a concept of Customer. That customer might have an account number.  You don’t want to call it “CustomerID”, because this is a human-generated number, rather than a DB auto-number.  Since you are following good design practices, you know better than to do that sort of thing.  You also want to make sure nobody else calls it that, especially in code or DB, etc.  So, you pick a good name to standardize on.  The Customer’s account number will always be called “AccountNumber”.  Then you enforce the use of that name in all of your tech/arch layers: DB, configs, screen fields, classes/interfaces, and everything else that an IT person would use it.  There are no exceptions allowed.

Your marketing or HR folks can come up with cute names like “the Hug Tag” or “Smile badge” or “Customer ID”.  Whatevs.  You and your team never call it by an unsanctioned name, when speaking with each other.  We know you cannot train the marketing folks, but you can train your own people.

The concept behind “Grave Naming” is that it is easier to find/trace data as it goes through a system if you use consistent standardized names.  Many companies do this sort of thing by creating their own local jargon.  The “Grave Naming” concept just takes the idea of jargon to a higher level, by applying it to IT and turning it into a form of standardization.  In the end, it makes things easier when you are consistent across all layers and components of your technology stack.

On paper, it sounds like a pretty good idea.  However, I imagine it could be pretty difficult to create a “Grave Naming” standard at some big companies.  Depending on how large or segmented your organization may be, it might be nearly impossible to enforce.  As a principal, I can see how it would make a huge difference (in a positive way) at large organizations who are currently segmented and suffering from the side-effects of it.  However, those people are more-likely suffering from the side-effects of their organization size and the natural fragmentation (and conflict) that commonly goes along with it.

So, from a strictly-academic standpoint, it is an excellent idea.  Now… if you can just convince your VP(s) to mandate that everyone follows this standard (because it saves time and money and reduces errors), then you have really got something good.

Remain steadfast, my fellow elitists.

Posted in Methodology, Programming | Tagged , , , , | Leave a comment

Showing Initiative vs Being a Cowboy

A few weeks ago, I fixed a bug without being asked to do so. Shortly afterwards, I was asked “Why did you fix that bug? Who told you to do it?”  The question arrived in email, so I couldn’t be certain about the tone-of-voice, but I’m sure you read this sentence with the same tone-of-voice that I did.  You are imagining someone giving you a sour-look or maybe even a scowl.

If you were a smart-aleck, you could respond like you didn’t know it was a rhetorical question “I fixed the bug because it was a bug. That is what I do when I find a bug. I fix it. I’m pretty sure you don’t want me to do the opposite of that.” However, since you and I are very professional, I wouldn’t say something like that. Right?

When I’m asked a question like this, it does bother me a little. Mostly, because I know my own motivations. I am not a loose cannon and I don’t go around looking for trouble.  I also don’t typically invent assignments for myself.  So, when I find a problem and I fix it without being told, I know I am being proactive, a.k.a. “showing initiative”.  I don’t expect a pat-on-the-back, but I certainly don’t expect some reprimand or questioned about my reasoning or motivation.  This was some kind of misunderstanding, and it just needed to be worked-out.

Cowboy “Fixes”

I have worked with a few cowboys in my day. Those guys can be fun to work-with, but they can also cause some interesting problems.  Therefore, many companies have processes in-place to limit the damage that a cowboy could cause.  Most notably is controlling the dev and release process, to monitor and control what gets worked-on and included in any release.

Now don’t get me wrong. Not every action from a cowboy is a dangerous one.  A cowboy mostly does good work.  It is the dangerous stuff that really draws consternation and earns the cowboy his dubious title.  So how does a person tell when an action is “cowboy-ish” or not?

Here are the conditions when the action of a developer is cowboy-ish (or generally, overstepping a boundary, or outright dangerous):

  • Security reasons – an IT department needs a check-and-balance, especially when it comes to security. It is not appropriate for anyone to modify security settings/configs, without regulation.
  • Stability – it only takes once or twice, where you trusted someone to apply a fix, and they broke your production servers. After that, you tend to not trust anyone as much. If you are responsible for the system, then be proactive and assess any changes, before they are applied.
  • Cost/scheduling – Let’s face it, everything takes time and time equals money. Developers shouldn’t approve their own schedules, or purchases, beyond a reasonable limit.
  • Control – Management is there for a reason. They need to be informed about what is going-on. There needs to be a way to track this. Whatever you do to track your work, it needs to contain enough detail to keep everyone honest. People make mistakes. Everyone needs to stand behind their work and own their successes and failures equally.

It is pretty common to put processes in-place to limit people from fiddling with any of these topics. Only your most trusted people should have permission to work in these circles.  Even then, your most trusted people still have to follow a process and have some reasonable limits and checks in-place.

Cowboy Detectors / Monitors

Placing limitations or verbal measures in-front-of the cowboy is usually not effective.  A cowboy is inclined to get around boundaries and hurdles without notice (sometimes, just for the challenge).  The only safe way to enforce your process is to steadily assume that the cowboy will periodically test the boundaries.  The real defense here, is to monitor and test, often.

Lets just assume for a minute, that not-all developers are cowboys. In that case, when it is right to make a change/fix without being prompted?

  • Trust smart people – face it, if you trust your developers to modify code, then you should also trust their judgment elsewhere. If you can’t trust them, then review their work periodically.  Find the line between management and micromanagement so you have control, without getting wasteful. After all, the programmers may be some of the smartest people you know. Certainly they must be trainable. So train them, and trust them a little. If you need more communication, then focus on that, rather than pulling the rope tighter.
  • Bureaucracy becomes too restrictive – This is why the agile manifesto was written. Sometimes well-meaning people get carried away with creating processes to protect the company and people.  Sometimes you need a fix ASAP.  Define a process for doing something quickly.  Of course it should be the exception. So define rules for when it is acceptable.
  • For the right reasons – back to my initial statement: a bug is a bug. Once you’ve found a bug, it usually takes seconds to fix it. However, it could take minutes or even hours, to log and track a bug and its fix. A team needs to find an efficient balance between regulation and waste.

In the end, I worked it out with my supervisor.  She just wanted to be informed about every bug that I find and the nature of the bug. That way, she could track the testing of the bug and add it to the statistics that went to management.  It was a very reasonable request and I was glad to contribute to the improvement of our process.

Posted in IT Psychology, Professionalism | Tagged , , , | Leave a comment