I recently ran into a rather annoying (confirmed) bug in an open source software package that I have integrated into my application. According to the public issue tracker this bug has been resolved in the latest release of the software.
Occasionally you NEED that bug fix to avoid an expensive refactor of a particular module, however for technical and/or political reasons, you will not be able to upgrade to the latest release.
On inspecting the code changes made, the fix seems simple enough that I feel a viable option would be to patch the code myself and recompile my current approved version of the software, however detractors want to argue the case that this is nearly always a bad idea in that it is risky and introducing a troublesome complexity.
In their eyes because this code change was done by us solely for our use, it must be part of our code base, meaning that rather than introducing the open source software as a third party dependency, we must introduce it as a new project and incorporate its automated build into our build process.
To me I think this is wrong headed, as we would be pulling their code from their source control repository into ours, and we lose the history behind any code changes that came before that. Also it just seems like something that is far too complicated for such a small code change that needs to be made.
Would it be a bad idea to do the above in this case? If so, then what is the ideal situation when open source needs to change, but only for your sole benefit in house?
2
If you can’t use a later version that doesn’t have the problem you’re encountering, the only options you have are to either
- live with the problem and find a workaround
- fork the library and fix it in your private version (which is what you’d effectively be doing)
- throw in the towel and tell your managers that the problem is insurmountable (which would be a lie, as you have two other options open to you).
I’ve been in your position, option 2 (make a custom fork) is often the most palatable solution available. That’s life when dealing with open source libraries, especially ones that evolve quickly and have a bad habit to break backwards compatibility between releases (which in my experience is the most common reason to have to do things like this).
For more than a few OSS libraries it’s led me and teams I’ve been a part of to mandate wrappers around any and all of them and accessing the functionality of 3rd party libraries exclusively through those wrappers. That way, if we need to replace a 3rd party library with a version that’s so different it’d break our code, the changes are at least largely confined to that wrapper. It’s not the nicest (adds code, can add complexity and cost performance) but sometimes it’s the only way to retain your sanity.
5
What you are about to do is a bad idea in the more common case where you bundle third-party software and intend to track their releases. Usually people do that because they want a feature in the third-party component that the maintainers aren’t willing to implement, or to implement in the way you need.
You, however, explicitly said that you will not upgrade the bundled code. That makes you effectively the maintainer of the third-party component. Therefore, whether patching it is a good idea or not depends only on whether you understand that code well enough to be confident of the desired effect. Your integration tests should be enough to verify that it is, in fact, doing what you assume. Therefore, as you tell the situation, it seems to me that your reviewers are wrong.
There’s really not anything wrong with doing that as long as everyone can stomach the costs, benefits and risks.
…the fix seems simple enough … to patch the code myself
When you have a job to do, perfect (having a third-party library that’s exactly what you want) is the enemy of good enough (patching it yourself), and sometimes you have to do things like that. I’ve done a number of projects where we’ve bought source licenses for commercial libraries so we could fix problems before the vendor got to it.
…detractors want to argue the case that this is nearly always
a bad idea in that it is risky and introducing a troublesome complexity.
It’s a bad idea if you don’t have the chops to handle dissecting someone else’s code, identifying a problem and writing a fix. That’s true whether the code is in-house or a third party; the only difference is whether it was thrown over a cubicle or building wall before it landed in your lap.
If your detractors are simply brushing the idea aside without weighing the costs of not doing this patch, they’re not doing their homework. If you have a lot of in-house code that’s affected by the bug your patch would fix, you’ll have to go through and change it to work around it and re-test everything to be sure it works correctly. Then, should you ever upgrade the package to a bug-fixed version, you may have to find and remove your workarounds and re-test again. There are risks to doing that as well, like missing a case you changed or insufficient testing. Personally, if I have the opportunity to fix a bug at its source, I’d much rather do it there than chase around the rest of the code with a flyswatter and hope I get everything.
…code change was done by us … it must be part of our code base
…we must introduce it as a new project and incorporate its
automated build into our build process.
If you’re doing a patch, the patch is part of your own code, which means you have to make it part of your process. This isn’t any different than adding something that’s 100% your code to your system. Treat the third-party distribution as sacrosanct and put it into a module just like it were source code. Any patches you write are stored with it in separate files and applied as part of the build process. That way you always go from clean source to patched source to built product and can show exactly what’s going on. (Some folks unpack, hand-patch, re-pack and store that in version control. That’s bad.)
…we would be pulling their code from their source control repository
into ours, and we lose the history behind any code changes…
If you’re treating the third-party library as a third-party dependency, you don’t have that history to begin with and you’re not losing anything. If you have continuing access to the third party’s repository, you can consult that should you need to. The third-party releases should be treated like amorphous blobs that you check into your own system unaltered. If you need to look at changes between the release you’re using and later releases, you can do that and, should you want to, come up with patches to the old version that incorporate changes you want.
Also it just seems like something that is far too complicated for
such a small code change that needs to be made.
If your build process is sufficiently sophisticated, adding this shouldn’t be any more difficult than adding your own code. There’s a small amount of labor in getting it to the point where the unpack/patch/build process is automagic, but once it’s done, it’s done forever. There may be one bug now, but there could be twenty in the future. If there are, you’ll be much happier that you laid the groundwork to support all of that now, because it will make dealing with the next 19 much less work.
0
What you want to do seems reasonable enough, but it sounds like there are (sound?) process reasons for opposing it. I won’t compare the proposed solutions, but perhaps there’s a way you could have your cake and eat it too:
If the open source project in question allows it, contribute your back-ported bugfix to their repository. That is, if you’re using version 1.5.2 and the current stable version is 1.6.1, contribute a patch to 1.5.2. If it gets adopted, you can fetch the fixed source directly from the repository (perhaps as version 1.5.3) and make everyone happy.
In other words: Patch it for everyone else who’s in your situation, too. Of course this is only possible if the project supports (or at least allows) updates to released versions. But that’s certainly pretty standard practice these days.