Whenever I work on some fairly significant piece of code, often I find that the I go through a phase of high level designing (aside: I like to use pencil and paper for this), followed by mental walkthroughs to highlight flaws and make refinements.
I see coding as another level (and finest grained) of mental walkthrough – you have to consider and cover every eventuality and this often results in further refinement of the design. Usually I find at the end of the process that enough has been learned that if I were designing and writing the code again there are things I could do better. And some of those things are perhaps too ‘baked in’ to the structure of the code to warrant spending the time to make the changes.
It often seems to me that in an ideal world I would always plan to go through the above process with the intention of using it as a learning process, and to plan to rewrite. Noting that much of the nuts and bolts of the code (e.g. low level subroutines) will be re-usable, so it’s not a full bottom up re-write.
3
Everything you say in your question is absolutely true, though I have a feeling that many companies don’t actually go through quite the same thought process as you.
Your thought process is (more or less, for any significant piece of code) “I need a day to write it, a day to debug it, and a day to refactor it.” Your company is saying “You need a day to write it.”
Your company knows that time to market is everything; a marginal piece of software that strikes gold first often wins market share over the highly-polished latecomer. It’s even worse than that; the first to market isn’t always the one that strikes gold; often, a company has to go through many iterations of a product before people like it and embrace it. King wrote almost a hundred games before Candy Crush came out, and Candy Crush represents something like 75% of their revenue.
Consequently, companies will almost always push to get the product out the door at the expense of clarity and elegance of code. It’s not the most comfortable position to be in as a software developer, but it is a challenge met by those who can balance the need to meet deadlines with the need to reduce technical debt as much as possible.
Also, remember that customer feedback is paramount. Unreleased software doesn’t get customer feedback and it doesn’t make any money. The sooner that a product gets released (consistent with stability and a reasonable feature set), the sooner you can find out what the customer really wants. This is the big reason that agile methodologies that get early customer feedback are now favored over “big design up front” methodologies like waterfall.
2
You are describing a core component of clean coding.
No matter how well you think you design something up front, development is an iterative process that demands change. You start with the high level overview of a system and design accordingly. Maybe you go through several ideas before settling on the right idea.
Then you start diving into slightly more detail. Maybe you see that your original design needs to be changed. Maybe it needs to be replaced entirely. This is a natural consequence of a top-down approach to design that is so common in our industry.
If you develop using an iterative approach (who uses Waterfall anymore?), the details necessarily move from coarse (early sprints) to detailed (later sprints) as you code more and more functionality. This may involve changing earlier code to work better with requirements implemented in later iterations.
This is natural and good: embrace the change.