The only difference between Amateur and Professional engineers

I believe it’s controversial, but I am still a firm believer in this idea.

When I was a university student, one of the classes that I was bored with was a Java programming. While most of the contents were a waste of time for me (because I was coding from 12 years old), I strongly remembered one straightforward word from the lecturer.

Do you guys know what’s the difference between pro and amateur?

I thought of skills or knowledge. I wanted to learn more about professional coding rather than such an easy thing.

The lecturer answered differently.

Professional engineering is all about Risk Anticipation

There’re tons of risks in engineering, and that made the quality issue very severely. Amateur engineers are good at only regular cases. In contrast, professional engineers always pay full attention to various types of risks, and this risk anticipation is 80% of the contribution of engineering, or even more.

Japan’s submarine technology

As a Japanese, I am proud to say that Japan’s submarine technology is currently the best in the world.

The ability of the submarines come from the noise; if it makes any noise, it’s easier for enemies to find the location, then it’s easily attacked. Therefore the engineering aspect of it is all from the risk anticipation and handling. Regardless of what happens, it shouldn’t make any noise. Shark attacks? Regardless, no sound.

What are the risks of coding?

There are several types of risks. Every professional coder must take all of the following aspects into consideration.

Student-level minimum

• No fatal error in regular cases
  Of course. Fatal errors are not acceptable, even on the student level.
• No deadlock in RDB
  Deadlock is so basic. All real software engineers must have this knowledge and understand how to avoid them.
• Consideration on Full-disk, Out-of-memory
  Full disk or out of memory is quite common reasons to get the script terminates. No real software engineers can ignore those risks.
• Timeout control (never blocking other tasks)
  Without timeout control, the software may easily go down.

Basic Level — common sense BELOW junior

• Any type of inputs are acceptable
  A little bit advance than the student level, yet this one is also very basics. We must assume any information.
• In any status of the systems, no error happens
  That’s why stateless architecture or status manager implementation happens.
• In any error case, it’s a traceable
  logging framework and designs.
• No middleware dependency risks
  We can’t guarantee that the middleware are accurately installed in the environment we define. We must assume that the middlewares may be different in others’ situations.
• Server health monitoring
  Servers may have some issues, such as disks, or CPU usage, or memory usage. Typically those problems must be well monitored.
• Security monitoring and attack detection
  In any security incidences, it’s 100% software engineers’ responsibility. Not only security incidence prevention, but we also have to monitor, and attacks must be detected.
• Browser dependency
  People use many types of browsers.
• Avoid Integration Risks (e.g., other API may stop)
  We shouldn’t believe anyone. Others make bugs. The software must run even with some API errors.
• No problems with many requests (e.g., Queue)
  Amateur codes don’t assume that the many applications may come. Professionals can write safer systems, maybe in queue architecture way.

Junior Level

• Spec may change (maintainability)
  Real Engineers with experiences start to understand that the specs are always changing. Therefore the codes must be written in the way change can easily be applied WITHOUT REGRESSION ERRORS.
• Traffic may increase (scalability)
  Real engineers can also assume how to make things scalable in the end. If we say, “The spec, for now, is only with one server,” but it may easily be changed. So, naively assuming some natural constraints are stupid.
• Recoverability from downtime
  Professionals don’t assume that the scripts can run forever. With many preparations, still, some problems may happen. How to recover from the error is the key.
• Migration may happen in the future
  Maybe the server will change. We must assume how to migrate to the environment in the future.
• The setting may change (logging, DB server, etc.)
  It’s often the case. Because of the other circumstances (e.g., security people want to monitor the DB and logs), the method of logging or DB access must be changed. Engineers must assume this thing will happen soon.
• Data access method may change (local disk to S3, etc.)
  Data access is one of the typical things which change continuously.
• 3rd party software may upgrade
  3rd party is continually updating. Without useful abstraction of the entrance to other software (API), it’s so chaotic if we apply the changes. Moreover, the monitoring of API spec changes is also essential.
• Regulation may change, and we have to comply (GDPR)
  Regulations can easily change. OK till yesterday could not be OK today. We must be aware of it.

Senior Level

• Junior members may do something wrong
  Even though you know the risk, maybe other members may not know the risk. Composition of the management flow (e.g., Unit Testing, CI, Pull Request, and Code Review) is the key.
• Codes may change which may cause other bugs
  If you manage all, it’s easier to handle. If others change the codes, and it connects to the bugs on the other guys’ codes, it’s quite chaotic. Architecturally solving those dependency problems is the key to seniors.
• Members may leave
  People may leave the company. Risks of unknown-ness must be managed appropriately.

Too many things to consider!

Overwhelming amount of things to consider It is the reality of software engineering. The way to avoid the risks is constraints.

The set of the definition of technical constraints is sometimes called “architecture” or “framework,” which is useful to skip your thoughts.

Reconsider technologies from a risk point of view

Many frameworks, architectures, middlewares are a bit more complicated than you feel comfortable with. It gives you a bit of stress when learning, and also it may increase the codes from what you imagine.

Let’s take an example. Amazon SES, the simple email service. It’s SUPER EASY to use SendMail, which can run locally. The use of SES is even harder.

Then why we should use it?

Because there’s a risk of being regarded as spam if we use SendMail.

What is the benefit of this technology?

Kubernetes? Oracle? Amazon RDS. Aurora. S3. Django, etc.

Some of the technology could be more benefit focus, but a lot of techniques are more on the risk focus. Therefore you shouldn’t focus on the benefit side of the technology.

Instead, you should try to question.

What is the risk this technology tries to avoid

Summary

Risk anticipation is the basis of the engineerings; otherwise, we can’t be professional as engineers.