Let’s talk about considering the implications
‘Implications’ are the expected (and unexpected) effects that are applied to your current situation, either after you change something or intend to change something about that situation.
As software professionals we all need to be better at considering implications when we propose an approach, design or technology, and as a software developer specifically hoping to move into an architecture role, it’s something that you’ll need to refine and hone.
Considering the implications is your spidey sense that warns you to be wary of jumping down a path; it’s the scars you’ve developed when things go wrong; it’s the wisdom you’ll have that saves you and your team pain in the future. Now for some examples.
The first is a case of Elastic Search. Elastic Search is a nice piece of technology, but I really find that people want to use it for everything. Need search? Lets just put in an ES cluster. Job done. Need a simple lookup website? Ah sure, forget using a database, they are so uncool, lets use Angular + ES. All the shiny. (As a side note, my Polish colleagues named this behaviour ‘CV driven development’ and you’ll typically see it with ‘fast and loose contractors’ that want the next best thing on their CV.)
So the example in this case was a requirement for search. An app already had been built which was a simple web based solution with a MySQL DB at the backend. There was a new requirement to search by a location, and then two further new requirements to search by 2 separate classifications, lets call them model and inspection.
The team looking after these requirements went ahead and put in Elastic Search, and ended up with a nice search page that even had fancy lookahead typing. The Product Owner loved it as it did the job very well, but no one outside the team knew that this had occurred.
Here’s the fun part — for each of those 3 search requirements, the production version of the system would need to have data migration jobs populate 22’000 rows for one search and 200 million and 300 million rows respectively for the other two search types. Bananas. You’d need a significant ES setup to handle the volumes, and then have on-going data migration jobs built to keep the indexes up to date. A simple task to build search in development turned into a massive task in downstream environment builds.
The lesson to learn here is that any design and development work needs to consider (and most importantly, have input from) Infrastructure, Testing, Ops, UX, QA, etc. You can’t build in a silo without understanding what production dependencies you’ll need to cover off. Think ‘what does production need’ and you can’t go far wrong.
Ultimately the whole piece of search work in the example I mentioned was paired back, with basic search against the database being implemented instead. It did the job — there were no additional requirements for Infrastructure or data migration, and the UX was consistent with what already was in place. Lookahead typing was ditched as it was a ‘nice to have’ and it was replaced by having the app ask the user to apply some filtering criteria to their search instead, which effectively made querying large volumes way more manageable.
Example 2 — External constraints being forgotten about
An external constraint is something usually outside the development teams control, and two big ones are time available, and budget. Of course, you have others like governance processes of the organisation, licensing, development skills needed, technology landscape of the organisation etc.
I’d been involved with a team that was looking at integration between two parties to move files from A to B using a pretty old school (but well established) approach of FTP transfer. The files being sent about were scanned images of paper files that are posted in.
Now I totally understand that this is not a new way of doing things, but one of the proposals floating about was to build a small microservice between the two parties in order to bridge the gap between them. A queue would hold JSON messages (image + metadata), and a background app would lift messages from the queue and submit them to a downstream system.
Sounds ok, but in effect we are increasing complexity significantly. We now have additional integration points between parties, additional failover and recovery considerations and additional bottlenecks when scaling. Furthermore, you have additional communications overhead between 3 parties, which in a time constrained project really mounts up, along with the task of aligning 3 separate development streams.
Now I don’t mind queues if they are used in the right way, but again, is it needed? In some circles you have governance processes around the technology used, and suddenly you’re going off and evaluating queue options and taking things to design authority boards. More time gone.
The funny thing is, is there really a user need for real time in the first place? Given that the documents are being posted in, with those same documents being subjected to the intricacies and delays inherent in postal delivery, why is there a need to have them submitted in real-time after they have been digitised? You’d be optimising the end journey of something that isn’t really optimisable as a process in its current form. I’d say that in effect, the process should really be to try and eliminate the need to send paper about in the first place, with a first class service providing users with a digital means of interacting with you. It’s solving the wrong problem.
So all I can say is that we all need to really consider all the implications when we propose a design, technology or approach. It’s not just a story that you need to finish for development, but a piece of a much larger solution that needs to be productionised as early as possible.
If in doubt, make a list of all the things you can think of that are good/bad/a dependency for your chosen problem, and then ask your team their thoughts, and then ask even more people. It will help shed some insight onto things, and at least you’ll make a better informed decision. Update — if you want a really good approach for doing this, check out my friend and colleague Rory‘s post: You must think first, before you move.
Don’t do CV driven development, but use the right technologies for the job. It might not be shiny, but if it meets the users need then they’ll appreciate you and your team more for it. Simplicity is better than shiny.
To read more of Davey’s blog, follow: https://medium.com/@daveym
Sign up to the Kainos newsletter