The first half is this essay asserts that software engineering is like a science because "software development is actually full of discovery" and not the "engineering practice of combining and assembling what is available from the complex system of computing in order to manifest a given vision."
This is extremely poor understanding. All engineering is full of discovery. Electrical engineers are constantly discovering new circuits that do the same task but with different resource usages and performance trade-offs. Civil engineers are doing the same thing with buildings. etc.
> Software development has the benefit of relying on abstraction layers....
So does every other engineering discipline. You think every new machine is built from first principles?
> Computing is so complex that these interfaces are never perfectly clean...
Author seems to imply that this is something unique to computing. Probably because they have never tried to build a complex analog circuit by assembling together a bunch of component circuits.
It's both the ego and arrogance of the software developer.
Like there's like nothing on people management in orgs. Like treating engineers like robots as though they'll unquestioningly follow what the author prescribed.
> a filmmaker can use a camera to make great films without understanding in detail how to build the camera, and there is probably not much of a predictable relationship between knowledge of how the camera works and quality of the resulting film.
This sentence conflates two different things to support a single point. One is true but the second is not. It's correct that a filmmaker does not need to know "how to build a camera" to make great films. However, a filmmaker certainly needs to know "how a camera works" to make a great film. And not just in the pedantic sense of where the Record button is but in the sense of a great violinist needing to know how their violin works (but not how to make a violin). The camera is the artistic instrument of cinema and using it well requires understanding how to leverage lens selection, aperture, shutter speed, exposure, focal plane, lighting, framing, etc to achieve the desired artistic outcome.
Another possible confusion in this sentence is the catch-all term "filmmaker" when the actual example is more narrowly focused on operation of the camera. Deciding how to apply a camera toward making a motion picture is generally the responsibility of the cinematographer. Sometimes uniquely skilled directors can also act as their own cinematographer but the roles and skill sets are as divergent as composer and violinist.
For anyone confused by this point perhaps watch some of the “why [movie] still looks like a billion bucks” videos on YouTube by Wolfcrow.
The idea that a film artist doesn’t need to know how a camera works is laughable. And not just the camera, but different film stock, processing methods, lenses, lighting, and the interaction of all those as well as the subject being shot. The level of attention to detail in great films is wild.
I once heard a great question to illustrate this... When you are inspired by a painting, is this your first thought:
"I need to figure out what brand of paints and brushes this painter used so I can paint like this."
Of course not! It's the painter that determines the quality of the painting and not the maker of the tools. A great painter (or filmmaker or cook or software engineer or ...) will understand the unique attributes of the tools available and leverage those to do great work, whether that's in oil or water, black & white or color, Python or Rust. A difference between experts and non-experts is that experts will find opportunity in the constraints of the tools they have available, and not only frustration.
I believe this extends to engineering managers. I want managers who are good at spotting the unique advantages of each person on the team and are good at structuring the roadmap and individual projects to let the team get the most out of those advantages. And I want them to spot where advantages and disadvantages can be paired to enable learning. All this helps teams see how valuable their teammates can be and how much they can learn from each other. I've seen it result in teams achieving remarkable outcomes while having a ton of fun doing it.
Scaling this up further... I want senior management that is in tune with the unique advantages of different parts of the org, and who then use that knowledge to make better decisions about direction. We can go in direction A, B, or C... that group over there is great at A, so let's pick that and give it to that group. Some see Conway's Law as a sort of "doom", but I've seen it used to the org's advantage with this kind of thinking.
>The idea that a film artist doesn’t need to know how a camera works is laughable. And not just the camera, but different film stock, processing methods, lenses, lighting, and the interaction of all those as well as the subject being shot. The level of attention to detail in great films is wild.
The Aliens documentary is really good for this. So much of the special effects on the film are in camera tricks.
Cameron is a really interesting person to study because he absolutely mastered every part of film making before deciding to go all in on Avatar and ubiquitous 3d effects.
> The people who create software generally refer to themselves as software engineers
Engineers follow processes and measure things. If a developer is not measuring things, most are not, then they are not engineering. If, at least, they are doing something new they are writing. Otherwise, they are typing. Software employs a lot of typists.
If you want to select for excellent engineers you only need to identify two qualities: conscientiousness and persistence. Conscientiousness is awareness of the world outside yourself and is negatively correlated with intelligence. Conscientiousness is where you find disciplined industriousness. Add a little bit of intelligence and you also get people that can follow complex instructions and achieve high precision. Creativity is the child of persistence and high intelligence. Creativity is achieved by trying many variations of a process and carefully discerning quality against some audience or metric.
This is an absolutely ridiculous take. If my more traditional engineering friend spends a day putting together plans based on existing documentation of parts, is he suddenly not doing engineering anymore? Do you expect him to run off to the supplier with a probe in hand and redo all their data sheets?
I think there’s more to creativity than persistence and high intelligence. What about Docker, or Terraform? Super simple ideas that as far as I can tell, were not the result of persistence, and were not uniquely intellectually taxing to invent.
I don’t think there’s a formula for creativity. Sometimes it happens, sometimes it doesn’t.
Thanks for sharing! I had never really thought about the difference in meaning between the words Science and Engineering in this context.
I’ve had discussions with coworkers before about how we should describe ourselves, but never quite in terms of actual job titles. I really like the black box analogy you mentioned. It reminded me of something I came across recently — Conway’s Law.
It's a bit annoying to see something I think is worth keeping on a website, only to find that the author has excluded their website from the Wayback Machine.
Now the content can't be stored easily in a reliable way. I would have to host it myself and make some solution to find it.
>the real opportunity lies in understanding that you’re not bound by the conventions that have been passed down as “standard.”
While in agreement with the author, I think Moxie is missing here that if you want external funding you are in fact mostly bound by conventions, conventions that most VCs and investors expect you to follow. It's possible to go beyond the normal funding routes as well but it narrows the path considerably.
Perhaps though, VCs will change their mindset about this as the AI-driven company becomes vogue and strengthens the perception of small orgs doing big things.
It's worth noting that Moxie is the same Moxie of Signal.
He's also an anarchist. I don't think he's in it for the money. It's also worth noting that several multibillion dollar organizations use the underlying software too. I'm saying he's highly experienced and has created a successful business. Even without VC funding!
So there's a difference:
Is your intent to build a high quality product that will make money?
Is your intent to make high income by selling a product?
Personally, I'd argue it's the job of the engineer to focus on the product. The money is the domain of the business people. I'd also argue that the former option is better for society and we're in a bad situation if the latter situation is dominating. The contention between business people and engineers is healthy. It creates balance. Our job is to make the product the best it can be, their job is to sell it. If we don't do our job they're more than happy to sell garbage as long as people will buy it. And remember, VCs don't care about you or the company past their exit. That's not healthy for something if you really care about the product. But it's fine if you care more about the money.
> Just like with software, where deep understanding is often the basis for discovery, an organization has to truly understand itself to be primed for innovation. When teams operate as impenetrable black boxes, vision becomes myopic, and potential withers. Despite having immense talent, the large engineering organizations of SV often structure themselves in ways that isolate expertise. As a result, they’re probably sitting on a huge amount of untapped potential – creative and technical ideas that never connect.
Then continues:
> If you’re building a company or an engineering team today, I think the real opportunity lies in understanding that you’re not bound by the conventions that have been passed down as “standard.”
It feels like there would be very few people who would have bota (a) a deep understanding of the different ways that software devs can be organized and (b) the conviction to “compose” a radically different organization based on that understanding. (Bryan Cantrill is popping into my head for some reason as someone who might fit this mold.)
The corollary of "Software systems reflect the organizational structure they are developed in", would be "Design your organization to match the structure of the software you want."
That provides a baseline rationale and direction for organizational innovation. And for continuing to adapt it.
Engineering often supplies the potential energy fueling vision and vice versa. ARPANET was very specifically engineered with the vision of a network that could survive nuclear attack. Then it evolved into the internet, which fueled a new vision of collaboration and discovery for students, hackers, and researchers. Who in turn created systems that became new business models, and the cycle goes on.
> I think there is something uniquely magical about software, and part of that magic might stem from this tension in how we define it.
Well, yes, but I'd phrase this differently. To me, there is something uniquely unprofessional about software, and part of this unprofessionalism stems from our lack of understanding.
If you do get a computer science degree, congratulations! You have more knowledge than 90% of the people I've worked with in my 20-year career. But even then you only have the "book knowledge"; you still lack the real-world knowledge that comes from practicing a profession. Tradespeople are required to have both the book knowledge, and the practical knowledge (from working under a master). This solves a lot of problems by filling in the gaps that books leave out or haven't updated to yet.
Software feels like magic because there's so much to know. Even for experts, knowing "how to do something right" isn't totally clear. I think the cause of the uncertainty is software engineering's lack of standardization.
Not long ago, the world had no standard units for making buildings. Nails, bolts, wooden beams, sizes of walls, framing layout, roof design, etc varied widely. Making a building strong enough not to fall over, be blown over, shaken apart, or burn down in 10 minutes, required over-building it, because you didn't actually know how to figure out if it would fall over or not. But now the component parts of a building are made to a set minimum standard. Building codes lay out exactly how you can construct a building so that it's safe to use. It's still not easy to construct a building, but at least we know exactly how to do it, and we can check that it was done correctly. In this way, an expert can know exactly how to build a building the right way. Because of this, construction isn't magic. In fact, it's often derided as a job for people who aren't smart, yet you have to be fairly skilled to do any part of it well!
Software engineering still lacks the certainty that physical engineering figured out long ago. Apparently software isn't important enough for people to need to quantify it yet. As is usual with humans, it takes a busload of kids getting creamed by someone speeding through an intersection for us to finally put stop signs in place. I expect after the next world war, when our society is brought to its knees by a simple computer virus, security will become standardized and required, with liability for the "engineers" who don't follow the protocol.
....none of this has to do with organizations, though. Nothing magic about organizing software engineers. Successful management strategies that work at other companies, also work in software companies. There are additional management strategies that can help the lowest level of work become more efficient (https://www.atlassian.com/agile), but those things require the boring, old, "normal" management to be done correctly first. This has been written about to death by people who've studied it all their lives. But software people stick to their software people blogs, and software people management strategies, and never branch out to see how the wider world works. Then they wonder why it all seems like magic.
> Software engineering still lacks the certainty that physical engineering figured out long ago. Apparently software isn't important enough
I think it is widely viewed as important enough, and every serious attempt at a new language or many software tools involve massive efforts (when their ecosystems are considered).
But one challenge is, we can't really measure a new language until its been put to use by a lot people, encouraging an ecosystem of libraries and tools, in a virtuous circle.
By the time that virtuous circle confirms that yes, A B and C are great improvements, its very difficult to take the next step and add D, E, and F that worked so well in another language.
It happens, but it tends to happen slowly and paid for with complexity that a new language with A, B, C, D, E and F at the start would not have had to pay. But getting a new language and ecosystem revved up is much harder than adding complexity. Until someone else with a great deal of passion and time takes another big gamble and succeeds.
Meanwhile, all that code in all those languages doesn't go away.
C++ complexity is where every language that never gives up, never surrenders, maintains backward compatibility, and continues folding in more good ideas, is slowly heading toward.
Its like trying to improve standards while building an "organically growing mountain terrain" type city, where the city has three main ground levels, covering vast areas of land, with smaller 2nd level areas on top of each, then smaller 3rd levels on all those 2nd level regions, etc. The different innovations happening in particular towers are very difficult for other towers built with other standards to adopt. But all the towers need to keep growing.
Someone says, lets start over with a new city, and it may be a much better city in many ways, but not all, and if enough people help they get a good enough first level up so more people start building there. But the same problem accrues.
The first half is this essay asserts that software engineering is like a science because "software development is actually full of discovery" and not the "engineering practice of combining and assembling what is available from the complex system of computing in order to manifest a given vision."
This is extremely poor understanding. All engineering is full of discovery. Electrical engineers are constantly discovering new circuits that do the same task but with different resource usages and performance trade-offs. Civil engineers are doing the same thing with buildings. etc.
> Software development has the benefit of relying on abstraction layers....
So does every other engineering discipline. You think every new machine is built from first principles?
> Computing is so complex that these interfaces are never perfectly clean...
Author seems to imply that this is something unique to computing. Probably because they have never tried to build a complex analog circuit by assembling together a bunch of component circuits.
It's both the ego and arrogance of the software developer.
Like there's like nothing on people management in orgs. Like treating engineers like robots as though they'll unquestioningly follow what the author prescribed.
> a filmmaker can use a camera to make great films without understanding in detail how to build the camera, and there is probably not much of a predictable relationship between knowledge of how the camera works and quality of the resulting film.
This sentence conflates two different things to support a single point. One is true but the second is not. It's correct that a filmmaker does not need to know "how to build a camera" to make great films. However, a filmmaker certainly needs to know "how a camera works" to make a great film. And not just in the pedantic sense of where the Record button is but in the sense of a great violinist needing to know how their violin works (but not how to make a violin). The camera is the artistic instrument of cinema and using it well requires understanding how to leverage lens selection, aperture, shutter speed, exposure, focal plane, lighting, framing, etc to achieve the desired artistic outcome.
Another possible confusion in this sentence is the catch-all term "filmmaker" when the actual example is more narrowly focused on operation of the camera. Deciding how to apply a camera toward making a motion picture is generally the responsibility of the cinematographer. Sometimes uniquely skilled directors can also act as their own cinematographer but the roles and skill sets are as divergent as composer and violinist.
For anyone confused by this point perhaps watch some of the “why [movie] still looks like a billion bucks” videos on YouTube by Wolfcrow.
The idea that a film artist doesn’t need to know how a camera works is laughable. And not just the camera, but different film stock, processing methods, lenses, lighting, and the interaction of all those as well as the subject being shot. The level of attention to detail in great films is wild.
I once heard a great question to illustrate this... When you are inspired by a painting, is this your first thought:
"I need to figure out what brand of paints and brushes this painter used so I can paint like this."
Of course not! It's the painter that determines the quality of the painting and not the maker of the tools. A great painter (or filmmaker or cook or software engineer or ...) will understand the unique attributes of the tools available and leverage those to do great work, whether that's in oil or water, black & white or color, Python or Rust. A difference between experts and non-experts is that experts will find opportunity in the constraints of the tools they have available, and not only frustration.
I believe this extends to engineering managers. I want managers who are good at spotting the unique advantages of each person on the team and are good at structuring the roadmap and individual projects to let the team get the most out of those advantages. And I want them to spot where advantages and disadvantages can be paired to enable learning. All this helps teams see how valuable their teammates can be and how much they can learn from each other. I've seen it result in teams achieving remarkable outcomes while having a ton of fun doing it.
Scaling this up further... I want senior management that is in tune with the unique advantages of different parts of the org, and who then use that knowledge to make better decisions about direction. We can go in direction A, B, or C... that group over there is great at A, so let's pick that and give it to that group. Some see Conway's Law as a sort of "doom", but I've seen it used to the org's advantage with this kind of thinking.
>The idea that a film artist doesn’t need to know how a camera works is laughable. And not just the camera, but different film stock, processing methods, lenses, lighting, and the interaction of all those as well as the subject being shot. The level of attention to detail in great films is wild.
The Aliens documentary is really good for this. So much of the special effects on the film are in camera tricks.
Cameron is a really interesting person to study because he absolutely mastered every part of film making before deciding to go all in on Avatar and ubiquitous 3d effects.
Yes but at some point you can stop understanding how it works. Maybe it wasn't the best example but I think picking it apart misses the larger point.
> The people who create software generally refer to themselves as software engineers
Engineers follow processes and measure things. If a developer is not measuring things, most are not, then they are not engineering. If, at least, they are doing something new they are writing. Otherwise, they are typing. Software employs a lot of typists.
If you want to select for excellent engineers you only need to identify two qualities: conscientiousness and persistence. Conscientiousness is awareness of the world outside yourself and is negatively correlated with intelligence. Conscientiousness is where you find disciplined industriousness. Add a little bit of intelligence and you also get people that can follow complex instructions and achieve high precision. Creativity is the child of persistence and high intelligence. Creativity is achieved by trying many variations of a process and carefully discerning quality against some audience or metric.
If you hire a bunch of clowns you get a circus.
This is an absolutely ridiculous take. If my more traditional engineering friend spends a day putting together plans based on existing documentation of parts, is he suddenly not doing engineering anymore? Do you expect him to run off to the supplier with a probe in hand and redo all their data sheets?
I think there’s more to creativity than persistence and high intelligence. What about Docker, or Terraform? Super simple ideas that as far as I can tell, were not the result of persistence, and were not uniquely intellectually taxing to invent.
I don’t think there’s a formula for creativity. Sometimes it happens, sometimes it doesn’t.
> Software employs a lot of typists.
We have many strong typists.
What I want are more static typists
Thanks for sharing! I had never really thought about the difference in meaning between the words Science and Engineering in this context.
I’ve had discussions with coworkers before about how we should describe ourselves, but never quite in terms of actual job titles. I really like the black box analogy you mentioned. It reminded me of something I came across recently — Conway’s Law.
It's a bit annoying to see something I think is worth keeping on a website, only to find that the author has excluded their website from the Wayback Machine. Now the content can't be stored easily in a reliable way. I would have to host it myself and make some solution to find it.
>the real opportunity lies in understanding that you’re not bound by the conventions that have been passed down as “standard.”
While in agreement with the author, I think Moxie is missing here that if you want external funding you are in fact mostly bound by conventions, conventions that most VCs and investors expect you to follow. It's possible to go beyond the normal funding routes as well but it narrows the path considerably.
Perhaps though, VCs will change their mindset about this as the AI-driven company becomes vogue and strengthens the perception of small orgs doing big things.
It's worth noting that Moxie is the same Moxie of Signal.
He's also an anarchist. I don't think he's in it for the money. It's also worth noting that several multibillion dollar organizations use the underlying software too. I'm saying he's highly experienced and has created a successful business. Even without VC funding!
So there's a difference:
Is your intent to build a high quality product that will make money?
Is your intent to make high income by selling a product?
Personally, I'd argue it's the job of the engineer to focus on the product. The money is the domain of the business people. I'd also argue that the former option is better for society and we're in a bad situation if the latter situation is dominating. The contention between business people and engineers is healthy. It creates balance. Our job is to make the product the best it can be, their job is to sell it. If we don't do our job they're more than happy to sell garbage as long as people will buy it. And remember, VCs don't care about you or the company past their exit. That's not healthy for something if you really care about the product. But it's fine if you care more about the money.
The author writes:
> Just like with software, where deep understanding is often the basis for discovery, an organization has to truly understand itself to be primed for innovation. When teams operate as impenetrable black boxes, vision becomes myopic, and potential withers. Despite having immense talent, the large engineering organizations of SV often structure themselves in ways that isolate expertise. As a result, they’re probably sitting on a huge amount of untapped potential – creative and technical ideas that never connect.
Then continues:
> If you’re building a company or an engineering team today, I think the real opportunity lies in understanding that you’re not bound by the conventions that have been passed down as “standard.”
It feels like there would be very few people who would have bota (a) a deep understanding of the different ways that software devs can be organized and (b) the conviction to “compose” a radically different organization based on that understanding. (Bryan Cantrill is popping into my head for some reason as someone who might fit this mold.)
The corollary of "Software systems reflect the organizational structure they are developed in", would be "Design your organization to match the structure of the software you want."
That provides a baseline rationale and direction for organizational innovation. And for continuing to adapt it.
Engineering often supplies the potential energy fueling vision and vice versa. ARPANET was very specifically engineered with the vision of a network that could survive nuclear attack. Then it evolved into the internet, which fueled a new vision of collaboration and discovery for students, hackers, and researchers. Who in turn created systems that became new business models, and the cycle goes on.
> I think there is something uniquely magical about software, and part of that magic might stem from this tension in how we define it.
Well, yes, but I'd phrase this differently. To me, there is something uniquely unprofessional about software, and part of this unprofessionalism stems from our lack of understanding.
If you do get a computer science degree, congratulations! You have more knowledge than 90% of the people I've worked with in my 20-year career. But even then you only have the "book knowledge"; you still lack the real-world knowledge that comes from practicing a profession. Tradespeople are required to have both the book knowledge, and the practical knowledge (from working under a master). This solves a lot of problems by filling in the gaps that books leave out or haven't updated to yet.
Software feels like magic because there's so much to know. Even for experts, knowing "how to do something right" isn't totally clear. I think the cause of the uncertainty is software engineering's lack of standardization.
Not long ago, the world had no standard units for making buildings. Nails, bolts, wooden beams, sizes of walls, framing layout, roof design, etc varied widely. Making a building strong enough not to fall over, be blown over, shaken apart, or burn down in 10 minutes, required over-building it, because you didn't actually know how to figure out if it would fall over or not. But now the component parts of a building are made to a set minimum standard. Building codes lay out exactly how you can construct a building so that it's safe to use. It's still not easy to construct a building, but at least we know exactly how to do it, and we can check that it was done correctly. In this way, an expert can know exactly how to build a building the right way. Because of this, construction isn't magic. In fact, it's often derided as a job for people who aren't smart, yet you have to be fairly skilled to do any part of it well!
Software engineering still lacks the certainty that physical engineering figured out long ago. Apparently software isn't important enough for people to need to quantify it yet. As is usual with humans, it takes a busload of kids getting creamed by someone speeding through an intersection for us to finally put stop signs in place. I expect after the next world war, when our society is brought to its knees by a simple computer virus, security will become standardized and required, with liability for the "engineers" who don't follow the protocol.
....none of this has to do with organizations, though. Nothing magic about organizing software engineers. Successful management strategies that work at other companies, also work in software companies. There are additional management strategies that can help the lowest level of work become more efficient (https://www.atlassian.com/agile), but those things require the boring, old, "normal" management to be done correctly first. This has been written about to death by people who've studied it all their lives. But software people stick to their software people blogs, and software people management strategies, and never branch out to see how the wider world works. Then they wonder why it all seems like magic.
> Software engineering still lacks the certainty that physical engineering figured out long ago. Apparently software isn't important enough
I think it is widely viewed as important enough, and every serious attempt at a new language or many software tools involve massive efforts (when their ecosystems are considered).
But one challenge is, we can't really measure a new language until its been put to use by a lot people, encouraging an ecosystem of libraries and tools, in a virtuous circle.
By the time that virtuous circle confirms that yes, A B and C are great improvements, its very difficult to take the next step and add D, E, and F that worked so well in another language.
It happens, but it tends to happen slowly and paid for with complexity that a new language with A, B, C, D, E and F at the start would not have had to pay. But getting a new language and ecosystem revved up is much harder than adding complexity. Until someone else with a great deal of passion and time takes another big gamble and succeeds.
Meanwhile, all that code in all those languages doesn't go away.
C++ complexity is where every language that never gives up, never surrenders, maintains backward compatibility, and continues folding in more good ideas, is slowly heading toward.
Its like trying to improve standards while building an "organically growing mountain terrain" type city, where the city has three main ground levels, covering vast areas of land, with smaller 2nd level areas on top of each, then smaller 3rd levels on all those 2nd level regions, etc. The different innovations happening in particular towers are very difficult for other towers built with other standards to adopt. But all the towers need to keep growing.
Someone says, lets start over with a new city, and it may be a much better city in many ways, but not all, and if enough people help they get a good enough first level up so more people start building there. But the same problem accrues.
[dead]
[flagged]