Shared environments: the problem is the people

Talk to anyone in engineering: devs, architects, managers. The answer is always the same. Nobody is happy with shared environments. Dev, test, stage, demo, homolog. The name doesn’t matter. Non-production environments are universally bad. That’s the reality of our industry.

I’ve never liked any shared environment at any company I’ve worked at. And I’ll ask you the same question: have you ever seen a non-production environment that actually worked well?

Now, here’s the curious part. If you ask someone what the solution is, the answer is almost always: “if we just had one more environment, then it would work.” And my question is: if it didn’t work with the environments that already exist, why would it work with the next one?

What are environments for?

Some argue that environments are necessary for testing. I’d say that’s the most expensive kind of testing there is. Why do you need a shared remote environment? Why can’t you test on your own machine? Why can’t you have a cheaper testing mechanism?

Of course, some scenarios are harder. A proprietary legacy system without source code, for example, is genuinely difficult to test locally. Not impossible, but more work.

The problem is that many people don’t know better ways to test and assume the only option is a shared environment. Good engineers run everything they can on their own machines and avoid shared environments at all costs.

One could argue that complex, long-running tests require shared environments. But if you can reproduce those scenarios locally, the shared environment becomes redundant. For Continuous Integration (CI), it makes sense to have a centralized environment running tests continuously. But that’s CI, not a collective playground.

What if you can’t run locally?

Then you have something that’s genuinely expensive to test. But before accepting that, ask yourself: are you sure there isn’t a Docker or Podman container that solves it? Often the solution exists, it’s just not part of the company’s daily workflow, or people don’t know how to make the change.

Take AWS as an example. One of the things that bothers me most is that you have to pay to learn. Several services simply don’t run on your machine: Bedrock, SQS, agent-core. You need an account and you’ll pay a bill just to do a POC.

But engineers always find a way. There’s an open source project called LocalStack that I’ve been using for years. It emulates AWS APIs locally. It’s not for production, it’s for testing. And it solves the problem.

What if I don’t have an API?

It’s common not to have APIs for every “state” you need to induce in a test. For example: how do you simulate someone buying 300 iPhones on an e-commerce platform to test a fraud scenario? If the service doesn’t expose that API, you create a test interface. An API that only exists in non-production environments, with the sole purpose of creating the state needed for the test.

Not having an API is an inconvenience, not a blocker. If you have the source code, just go ahead and add the test interface you need.

What if I don’t have the source code?

Say it’s a proprietary legacy system with no APIs, just a graphical interface. Even if it’s desktop software, if there’s a UI where you can create the state you need through clicks, you can write software that does those clicks for you. Is it slower? Yes. But you can still automate the creation of the required state.

How to improve

Some practices that make a real difference:

• Automate all tests. If it’s manual, it will break.
• Isolate your tests. One test can’t depend on the state another left behind.
• Have real CI. Stop using feature branches as a crutch.
• Create test interfaces in services that don’t expose APIs for state induction.
• Prioritize local tests. The cheaper the test, the more you run it.
• Treat environments as immutable. Don’t deploy your branch to the dev environment. Don’t manually touch anything. Everything through automation.
• Talk about behavior. If the team doesn’t discuss how people use environments (usually in retrospectives), you can’t fix bad habits.
• Train managers to identify and correct behaviors that degrade environments.

Stop creating new environments. Start changing the culture and behavior of the people who use the ones that already exist.