There is no perfect code, no perfect language, no perfect framework or methodology. Everything is, in some way, flawed.
This realization can be liberating -- when you accept that everything is flawed in some way, it puts aside petty arguments about the best language, editor, IDE, framework, database -- whatever some zealot is pedaling as being the ultimate solution to your problems. It also illuminates the need for all of the different options out there -- new programming languages are born sometimes on a lark, but often because the creator wanted to express themselves and their logical intent better than they could with the languages that they knew. Or perhaps they wanted to remove some of the complexity associated with common tasks (such as memory allocation and freeing or asynchronous logic). Whatever the reason, there is (usually) a valid one.
The same can be said for frameworks, database, libraries -- you name it. Yes, even the precious pearls that you've hand-crafted into existence.
In fact, especially those. We can be blind to the imperfections in our own creations, sometimes it's just ego in the way. Sometimes it's just a blind spot. Sometimes it's because we tie our own self-value to the things we create.
"You are not your job, you're not how much money
you have in the bank. You are not the car you drive. You're not the
contents of your wallet. You are not your fucking khakis."
For the few that didn't recognize the above, it was uttered by Tyler Durden from the cult classic Fight Club. There are many highlights one could pick from that movie, but this one has been on my mind recently. It refers to how people define themselves by their possessions, finding their identity in material items which are, ultimately, transient.
Much like how we're transient, ultimately unimportant in the grand scale of space and time that surrounds us. Like our possessions, we're just star stuff, on loan from the universe whilst we experience a tiny fraction of it.
I'd like to add another item to the list above:
You are not the code you have written.
This may be difficult to digest. It may stick in your throat, but some freedom comes in accepting this.
As a developer, you have to be continually learning, continually improving, just to remain marginally relevant in the vast expanse of languages, technologies, frameworks, companies and problems in the virtual sea in which our tiny domains float. If you're not learning, you're left behind the curve. Stagnation is a slow death which can be escaped by shaking up your entire world (eg forcing learning by hopping to a new company) or embraced by simply accepting it as your fate as you move into management, doomed to observe others actually living out their passions as they create new things and you report on it. From creator to observer. You may be able to balance this for a while, you may even have the satisfaction of moving the pieces on the board, but you've given up something of yourself, some part of your creator spirit. You can still learn here -- but you can also survive just fine as you are. Or at least that's how it looks from the outside. And that's pretty-much how I've heard it described from the inside. I wouldn't know, personally. I'm too afraid to try. I like making things.
This journey of constant learning and improvement probably applies to other professions, especially those requiring some level of creativity and craftsmanship from the member. You either evolve to continue creating or your fade away into obscurity.
And if you are passionate about what you're doing, if you are continually learning, continually hungry to be better at what you do, continually looking for ways to evolve your thought processes and code, then inevitably, you have to look back on your creations of the past and feel...
Displeased.
Often I can add other emotions to the mix: embarrassed, appalled, even loathing. But at the very least, looking back on something you've created, you should be able to see how much better you'd be able to do it now. This doesn't mean that your past creations have no value -- especially if they are actually useful and in use. It just means that a natural part of continual evolution is the realization that everything you've ever done, everything you ever will do, given enough distance of time, upon reflection, sucks.
It starts when you recognize that code you wrote a decade ago sucks. It grows as you realize that code you wrote 5 years, even 2 years ago sucks. It crescendos as you realize that the code you wrote 6 months ago sucks -- indeed, even the code you wrote a fortnight ago could be done better with what you've learned in the last sprint.
It's not a bad thing. The realization allows you to divorce your self-worth from your creations. If anything, you could glean some of your identity from the improvements you've been able to make throughout your career. Because, if you realize that your past creations, in some way or another, all suck, if you realize that this truth will come to pass for your current and future creations, then you have to also come to conclusion that realizing deficiencies in your past accomplishments highlights your own personal evolution.
If
you look back over your past code and you don't feel some kind of
disappointment, if you can't point out the flaws in your prior works,
then I'd have to conclude that you're either stagnating or you're
deluding yourself -- perhaps out of pride, perhaps because you've
attached your self-worth to the creations you've made. Neither is a good place to be -- you're either becoming irrelevant or you're unteachable and you will become irrelevant.
If you can come to accept this as truth, it also means that you can accept criticism with valid reasoning as an opportunity to learn instead of an attack on your character.
I
watched a video where the speaker posits that code has two audiences:
the compiler and your co-workers. The compiler doesn't care about style
or readability. The compiler simply cares about syntactical correctness.
I've tended to take this a little further with the mantra:
Code is for co-workers, not compilers.
I
can't claim to be the original author -- but I also can't remember
where I read it. In this case, it does boil down to a simple truth: if
your code is difficult for someone else to extend and maintain, it may
not be the sparkling gem you think it is. If a junior tasked with
updating the code gets lost, calls on a senior for help and that senior
can't grok your code -- it's not great. If that senior points that out,
then they are doing their job as the primary audience of that code. This
is not a place for conflict -- it is a place for learning. Yes, sometimes code is just complex. Sometimes language or framework features are not obvious to an outside programmer looking in. But it's unusual to find complex code which can't be made understandable. Einstein said:
“If you can't explain it to a six year old, you don't understand it yourself.”
And I'd say this extends to your code -- if your co-workers can't extend it, learn of the (potentially complex) domain, work with the code you've made, then you're missing a fundamental reason for the code to exist: explaining the domain to others through modelling it and solving problems within it.
Working with people who haven't accepted this is difficult -- you can't point out flaws that need to be fixed or, in extreme cases, even fix them without inciting their wrath. You end up having to guerilla-code to resolve code issues or just bite your tongue as you quietly sweep up after them. Or worse -- working around the deficiencies in their code because they insist you depend on it whilst simultaneously denying you the facility to better it.
At some point, we probably all felt the pang when someone pointed out a flaw in our code. Hopefully, as we get older and wiser, this falls away. Personally, I think that divorcing your self-image from your creations, allowing yourself to be critical of the things you've made -- this is one of the marks of maturity that defines the difference between senior developer and junior developer. Not to say that a junior can't master this already -- more to say that I question the "seniority" of a senior who can't do this. It's just one of the skills you need to progress. Like typing or learning new languages.
All of this isn't to say that you can't take pride in your work or that there's no point trying to do your best. We're on this roundabout trying to learn more, to evolve, to be better. You can only get better from a place where you were worse. Once of the best sources of learning is contemplated failure. You can also feel pride in the good parts of what you've created, as long as that is tempered by a realistic, open view on the not-so-good parts.
You may even like something you've made, for a while, at least. I'm currently in a bit of a honeymoon phase with NExpect (GitHub, Nuget): it's letting me express myself better in tests, it's providing value to handful of other people -- but this too shall pass. At some point, I'm going to look at the code and wonder what I was thinking. I'm going to see a more elegant solution, and I'm going to see the inadequacies of the code. Indeed, I've already experienced this in part -- but it's been overshadowed by the positive stuff that I've experienced, so I'm not quite at the loathing state yet.
You are not your fucking code. When it's faults become obvious, have the grace to learn instead of being offended.
Saturday, 25 March 2017
C# Evolution
(and why you should care)
I may be a bit of a programming language nut. I find different languages interesting not just because they are semantically different or because they offer different features or even because they just look pretty (I'm lookin' at you, Python), but because they can teach us new things about the languages we already use.
I'm of the opinion that no technology with any staying power has no value to offer. In other words, if the tech has managed to stay around for some time, there must be something there. You can hate on VB6 as much as you want, but there has to be something there to have made it dominate desktop programming for the years that it did. That's not the focus of this discussion though.
Similarly, when new languages emerge, instead of just rolling my eyes and uttering something along the lines of "Another programming language? Why? And who honestly cares?", I prefer to take a step back and have a good long look. Creating a new language and the ecosystem required to interpret or compile that language is a non-trivial task. It takes a lot of time and effort, so even when a language seems completely impractical, I like to consider why someone might spend the time creating it. Sure, there are outliers where the only reason is "because I could" (I'm sure Shakespeare has to be one of those) or "because I hate everyone else" (Brainfuck, Whitespace and my favorite to troll on, Perl -- I jest, because Perl is a powerhouse; I just haven't ever seen a program written in Perl which didn't make me cringe, though Larry promises that is supposed to change with Perl 6).
Most languages are born because someone found the language they were dealing with was getting in the way of getting things done. A great example is Go, which was dreamed up by a bunch of smart programmers who had been doing this programming thing for a while and really just wanted to make an ecosystem which would help them to get stuff done in a multi-core world without having to watch out for silly shit like deadlocks and parallel memory management. Not that you can't hurt yourself in even the most well-designed language (indeed, if you definitely can't hurt yourself in the language you're using, you're probably bound and gagged by it -- but I'm not here to judge what you're into).
Along the same lines, it's interesting to watch the evolution of languages, especially as languages evolve out of fairly mundane, suit-and-tie beginnings. I feel like C# has done that -- and will continue to do so. Yes, a lot of cool features aren't unique or even original -- deconstruction in C#7 has been around for ages in Python, for example -- but that doesn't make them any less valuable.
I'm going to skip some of the earlier iterations and start at where I think it's interesting: C#5. Please note that this post is (obviously) opinion. Whilst I'll try to cover all of the feature changes that I can dig up, I'm most likely going to focus on the ones which I think provide the programmer the most benefit.
C#5
C#5 brought async features and caller information. Let's examine the latter before I offer up what will probably be an unpopular opinion on the former.
Caller information allowed providing attributes on optional functional parameters such that if the caller didn't provide a value, the called code could glean
Caller file path
Caller line number
Caller member name
This is a bonus for logging and debugging of complex systems, but also allowed tricks whereby the name of a property could be automagically passed into, for example, an MVVM framework method call in your WPF app. This makes refactor-renaming easier and removes some magic strings, not to mention making debug logging way way simpler. Not ground-breaking, but certainly [CallerMemberName] became a friend of the WPF developer. A better solution for the exact problem of property names and framework calls came with nameof in C#6, but CallerMember* attributes are still a good thing.
C# 5 also brought async/await. On the surface, this seems like a great idea. If only C#'s async/await worked anything like the async/await in Typescript, where, under the hood, there's just a regular old promise and no hidden bullshit. C#'s async/await looks like it's just going to be Tasks and some compiler magic under the hood, but there's that attached context which comes back to bite asses more often than a mosquito with a scat fetish. There are some times when things will do exactly what you expect and other times when they won't, just because you don't know enough about the underlying system.
That's the biggest problem with async/await, in my opinion: it looks like syntactic sugar to take away the pain of parallel computing from the unwashed masses but ends up just being trickier than having to learn how to do actual multi-threading. There's also the fickle task scheduler which may decide 8 cores doesn't mean 8 parallel tasks -- but that's OK: you can swap that out for your own task scheduler... as long as you understand enough of the underlying system, again (as this test code demonstrates).
Like many problems that arise out of async / parallel programming, tracking down the cause of sporadic issues in code is non-trivial. I had a bit of code which would always fail the first time, and then work like a charm -- until I figured out it was the context that was causing issues, so I forced a null context and the problem stopped. The developer has to start learning about task continuation options and start caring about how external libraries do things. And many developers aren't even aware of when it's appropriate to use async/await, opting to use it everywhere and just add overhead to something which really didn't need to be async, like most web api controllers. Async/await makes a lot of sense in GUI applications though.
Having async/await around IO-bound stuff in web api calls may be good for high-volume web requests because it allows the worker thread to be re-assigned to handle another request. I have yet to see an actual benchmark showing better web performance from simply switching to async/await for all calls.
The time you probably most want to use it is for concurrent IO to shorten the overall request time for a single request. Some thought has to go into this though -- handling too requests concurrently may just end up with many requests timing out instead of a few requests being given a quick 503, indicating that the application needs some help with scaling. In other words, simply peppering your code with async/await could result in no net gain, especially if the code being awaited is hitting the same resource (eg, your database).
Which leads to the second part of C#'s async/await that I hate: the async zombie apocalypse. Because asking the result of a method marked async to just .Wait() is suicide (I hope you know it is; please, please, please don't do that), async/await patterns tend to propagate throughout code until everything is awaiting some async function. It's the Walking Dead, traipsing through your code, leaving little async/await turds wherever they go.
You can use ConfigureAwait() to get around deadlocking on the context selected for your async code -- but you must remember to apply it to all async results if you're trying to "un-async" a block of code. You can also set the synchronization context (I suggest the useful value of null). Like the former workaround, it's hardly elegant.
As much as I hate (fear?) async/await, there are places where it makes the code clearer and easier to deal with. Mostly in desktop applications, under event-handling code. It's not all bad, but the concept has been over-hyped and over-used (and poorly at that -- like EF's SaveChangesAsync(), which you might think, being async, is thread-safe, and you'd be DEAD WRONG).
Let's leave it here: use async/await when it provides obvious value. Question it's use at every turn. For every person who loves async/await, there's a virtual alphabet soup of blogs explaining how to get around some esoteric failure brought on by the feature. As with multi-threading in C/C++: "use with caution".
Read-only auto properties made programming read-only properties just a little quicker, especially when the property values came from the constructor. So code like:
public class VideoGamePlumber
{
public string Name { get { return _name; } }
private string _name;
public VideoGamePlumber(string name)
{
_name = name;
}
}
became:
public class VideoGamePlumber
{
public string Name { get; private set; }
public VideoGamePlumber(string name)
{
Name = name;
}
}
but that still leaves the Name property open for change within the VideoGamePlumber class, so better would be the C#6 variant:
public class VideoGamePlumber
{
public string Name { get; }
public VideoGamePlumber(string name)
{
Name = name;
}
}
The Name property can only be set from within the constructor. Unless, of course, you resort to reflection, since the mutability of Name is enforced by the compiler, not the runtime. But I didn't tell you that.
Auto-property initializers seem quite convenient, but I'll admit that I rarely use them, I think primarily because the times that I want to initialize outside of the constructor, I generally want a private (or protected) field, and when I want to set a property at construction time, it's probably getting it's value from a constructor parameter. I don't hate the feature (at all), just don't use it much. Still, if you wanted to:
public class Doggie
{
public property Name { get; set; }
public Doggie()
{
Name = "Rex"; // set default dog name
}
}
becomes:
public class Doggie
{
public property Name { get; set; } = "Rex";
}
You can combine this with the read-only property syntax if you like:
public class Doggie
{
public property Name { get; } = "Rex";
}
but then all doggies are called Rex (which is quite presumptuous) and you really should have just used a constant, which you can't modify through reflection.
Expression-bodied function members can provide a succinct syntax for a read-only, calculated property. However, I use them sparingly because anything beyond a very simple "calculation", eg:
public class Person
{
public string FirstName { get; }
public string LastName { get; }
public string FullName => $"{FirstName} {LastName}";
public Person(string firstName, string lastName)
{
FirstName = firstName;
LastName = lastName;
}
}
starts to get long and more difficult to read; though that argument is simply countered by having a property like:
public class Business
{
public string StreetAddress { get; }
public string Suburb { get; }
public string Town { get; }
public string PostalCode { get; }
public string PostalAddress => GetPostalAddress();
public Business(string streetAddress, string suburb, string Town, string postalCode)
{
StreetAddress = streetAddress;
Suburb = suburb;
Town = town;
PostalCode = postalCode
}
private string GetAddress()
{
return string.Join("\n", new[]
{
StreetAddress,
Suburb,
Town,
PostalCode
});
}
}
Where the logic for generating the full address from the many bits of involved data is tucked away in a more readable method and the property itself becomes syntactic sugar, looking less clunky than just exposing the GetAddress method.
Index initializers provide a neater syntax for initializing Dictionaries, for example:
var webErrors = new Dictionary()
{
{ 404, "Page Not Found" },
{ 302, "Page moved, but left a forwarding address" },
{ 500, "The web server can't come out to play today" }
};
Can be written as:
private Dictionary webErrors = new Dictionary
{
[404] = "Page not Found",
[302] = "Page moved, but left a forwarding address.",
[500] = "The web server can't come out to play today."
};
It's not ground-breaking, but I find it a little more pleasing on the eye.
Other stuff that I hardly use includes:
Extension Add methods for collection initializers allow your custom collections to be initialized like standard ones. Not a feature I've ever used because I haven't had the need to write a custom collection.
Improved overload resolution reduced the number of times I shook my fist at the complainer compiler. Ok, so I technically use this, but this is one of those features that, when it's working, you don't even realise it.
Exception filters made exception handling more expressive and easier to read.
On to the good bits (that I regularly use) though:
using static made using static functions so much neater -- as if they were part of the class you were currently working in. I don't push static functions in general because using them means that testing anything which uses them has to test them too, but there are places where they make sense. One is in RandomValueGen from PeanutButter.RandomGenerators, a class which provides functions to generate random data for testing purposes. A static import means you no longer have to mention the RandomValueGen class throughout your test code:
using PeanutButter.RandomGenerators;
namespace Bovine.Tests
{
[TestFixture]
public class TestCows
{
[Test]
public void Moo_ShouldBeAnnoying()
{
// Arrange
var cow = new Cow()
{
Name = RandomValueGen.GetRandomString(),
Gender = RandomValueGen.GetRandom(),
Age = RandomValueGen.GetRandomInt()
};
// ...
}
}
}
Can become:
using static PeanutButter.RandomGenerators.RandomValueGen;
namespace Bovine.Tests
{
[TestFixture]
public class TestCows
{
[Test]
public void Moo_ShouldBeAnnoying()
{
// Arrange
var cow = new Cow()
{
Name = GetRandomString(),
Gender = GetRandom(),
Age = GetRandomInt()
};
// ...
}
}
}
Which is way more readable simply because there's less unnecessary cruft in there. At the point of reading (and writing) the test, source library and class for random values is not only irrelevant and unnecessary -- it's just plain noisy and ugly.
if (thing != null &&
thing.OtherThing != null &&
thing.OtherThing.FavoriteChild != null &&
// ... and so on, and so forth, turtles all the way down
// until, eventually
this.OtherThing.FavoriteChild.Dog.Collar.Spike.Metal.Manufacturer.Name != null)
{
return this.OtherThing.FavoriteChild.Dog.Collar.Spike.Metal.Manufacturer.Name;
}
return "Unknown manufacturer";
public void PrintHello(string salutation, string firstName, string lastName)
{
Console.WriteLine($"Hello, {salutation} {firstName} of the house {lastName}");
}
nameof is also pretty cool, not just for making your constructor null-checks impervious to refactoring:
public class Person
{
public Person(string name)
{
if (name == null) throw new ArgumentNullException(nameof(name));
}
}
(if you're into constructor-time null-checks) but also for using test case sources in NUnit:
[Test, TestCaseSource(nameof(DivideCases))]
public void DivideTest(int n, int d, int q)
{
Assert.AreEqual( q, n / d );
}
static object[] DivideCases =
{
new object[] { 12, 3, 4 },
new object[] { 12, 2, 6 },
new object[] { 12, 4, 3 }
};
C#7
This iteration of the language brings some neat features for making code more succinct and easier to grok at a glance.
Inline declaration of out variables makes using methods with out variables a little prettier. This is not a reason to start using out parameters: I still think that there's normally a better way to do whatever it is that you're trying to achieve with them and use of out and ref parameters is, for me, a warning signal in the code of a place where something unexpected could happen. In particular, using out parameters for methods can make the methods really clunky because you have to set them before returning, making quick returns less elegant. Part of me would have liked them to be set to the default value of the type instead, but I understand the rationale behind the compiler not permitting a return before setting an out parameter: it's far too easy to forget to set it and end up with strange behavior.
I think I can count on one hand the number of times I've written a method with an out or ref parameter and I couldn't even point you at any of them. I totally see the point of ref parameters for high-performance code where it makes sense (like manipulating sound or image data). I just really think that when you use out or ref, you should always ask yourself "is there another way to do this?". Anyway, my opinions on the subject aside, there are times when you have to interact with code not under your control and that code uses out params, for example:
public void PrintNumeric(string input)
{
int result;
if (int.TryParse(input, out result))
{
Console.WriteLine($"Your number is: {input}");
}
else
{
Console.WriteLine($"'{input}' is not a number )':");
}
}
becomes:
public void PrintNumeric(string input)
{
if (int.TryParse(input, out int result))
{
Console.WriteLine($"Your number is: {input}");
}
else
{
Console.WriteLine($"'{input}' is not a number )':");
}
}
It's a subtle change, but if you have to use out parameters enough, it becomes a lot more convenient, less noisy.
Similarly ref locals and returns, where refs are appropriate, can make code much cleaner. The general use-case for these is to return a reference to a non-reference type for performance reasons, for example when you have a large set of ints, bytes, or structs and would like to pass off to another method to find the element you're interested in before modifying it. Instead of the finder returning an index and the outer call re-referencing into the array, the finder method can simply return a reference to the found element so the outer call can do whatever manipulations it needs to. I can see the use case for performance reasons in audio and image processing as well as large sets of structs of data. The example linked above is quite comprehensive and the usage is for more advanced code, so I'm not going to rehash it here.
public static class TippleTuple
{
public static void Main()
{
// good
(int first, int second) = MakeAnIntsTuple();
Console.WriteLine($"first: {first}, second: {second}");
// better
var (flag, message) = MakeAMixedTuple();
Console.WriteLine($"first: {flag}, second: {message}");
}
public static (int first, int second) MakeAnIntsTuple()
{
return (1, 2);
}
public static (bool flag, string name) MakeAMixedTuple()
{
return (true, "Moo");
}
}
Some notes though: the release notes say that you should need to install the System.ValueTuple nuget package to support this feature in VS2015 and lower, but I found that I needed to install the package on VS2017 too. Resharper still doesn't have a clue about the feature as of 2016.3.2, so usages within the interpolated strings above are highlighted as errors. Still, the program above compiles and is way more elegant than using Tuple<> generic types. It's very clever that language features can be delivered by nuget packages though.
Local functions provide a mechanism for keeping little bits of re-used code local within a method. In the past, I've used a Func variable where I've had a little piece of re-usable logic:
private NameValueCollection CreateRandomSettings()
{
var result = new NameValueCollection();
Func randInt = () => RandomValueGen.GetRandomInt().ToString();
result["MaxSendAttempts"] = randInt();
result["BackoffIntervalInMinutes"] = randInt();
result["BackoffMultiplier"] = randInt();
result["PurgeMessageWithAgeInDays"] = randInt();
return result;
}
which can become:
public static NameValueCollection GetSomeNamedNumericStrings()
{
var result = new NameValueCollection();
result["MaxSendAttempts"] = RandInt();
result["BackoffIntervalInMinutes"] = RandInt();
result["BackoffMultiplier"] = RandInt();
result["PurgeMessageWithAgeInDays"] = RandInt();
return result;
string RandInt () => GetRandomInt().ToString();
}
Which, I think, is neater. There's also possibly a performance benefit: every time the first implementation of GetSomeNamedNumericStrings is called, the randInt Func is instantiated, where the second implementation has the function compiled at compile-time, baked into the resultant assembly. Whilst I wouldn't put it past the compiler or JIT to do clever optimisations for the first implementation, I wouldn't expect it.
public int ThrowIfNullResult(Func source)
{
var result = source();
if (result == null)
throw new InvalidOperationException("Null result is not allowed");
return result;
}
can become:
public int ThrowIfNullResult(Func source)
{
return source() ??
throw new InvalidOperationException("Null result is not allowd");
}
So now you can actually write a program where every method has one return statement and nothing else.
Time for the big one: Pattern Matching. This is a language feature native to F# and anyone who has done some C#-F# crossover has whined about how it's missing from C#, including me. Pattern matching not only elevates the rather bland C# switch statement from having constant-only cases to non-constant ones:
public static string InterpretInput(string input)
{
switch (input)
{
case string now when now == DateTime.Now.ToString("yyyy/mm/dd"):
return "Today's date";
default:
return "Unknown input";
}
}
It allows type matching:
public static void Main()
{
var animals = new List()
{
new Dog() {Name = "Rover"},
new Cat() {Name = "Grumplestiltskin"},
new Lizard(),
new MoleRat()
};
foreach (var animal in animals)
PrintAnimalName(animal);
}
public static void PrintAnimalName(Animal animal)
{
switch (animal)
{
case Dog dog:
Console.WriteLine($"{dog.Name} is a {dog.GetType().Name}");
break;
case Cat cat:
Console.WriteLine($"{cat.Name} is a {cat.GetType().Name}");
break;
case Lizard _:
Console.WriteLine("Lizards have no name");
break;
default:
Console.WriteLine($"{animal.GetType().Name} is mystery meat");
break;
}
}
Hark! 'Tis the call of a higher-order language. It beckons with sweet words of reduced time solving ancient, low-level problems! The siren speaks true!
Seriously though...
Higher-order programming languages afford the competent programmer a method for solving more higher-level problems in the same timeframe than she could in with a lower-order language. Simply put, high-order languages (like C++, C#, Python, Ruby, PHP, etc) afford the programmer the ability to skip over the low-level "move this byte to this address" kind of programming so that she can solve more interesting and/or more lucrative problems.
There is a cost though: iron. These higher-order languages require smarter compilers and, typically, more RAM and CPU. It's a cost we're happy to pay though -- iron is cheaper than development.
But when someone starts making outlandish claims that a higher-order language is more proficient than a lower-order one (https://isocpp.org/blog/2014/12/myths-1, see the string concatenation example in the first section), the reasonable programmer doesn't just gulp that down, even if it does come from the father of C++. Actually, especially if it does come from the father of a higher-order language, since he would have reason to pad out his results (or pretend to be lazy) to make his prodigy all the more appealing.
Aside: before we go any further, a disclaimer: I like C++. I like C. That's OK to like both and recognise their strengths. What's NOT ok (imo), is to spread misinformation to highlight the language of your preference. Have honest reasons for preference -- by all means! -- and be objective about comparisions. And the discourse continues!
So let me make a really bold statement:
Any proficient code in a higher-order language can only hope to be (at best) as proficient as proficient code in a language of lower-order for solving the same problem.
Why?
Let's take the string vs char* example from above:
std::string has to be implemented (at some point) around a buffer of memory. I don't care if it's char*, wchar* or whatever. It's a buffer which, at some point, was obtained via malloc() (even if you want to say it was obtained by new char[], that still boils down to essentially a malloc, so let's stop arguing semantics). The C++ compiler affords us the ability to overload operators, such that we can do:
string1 + string2
and get another string. Under the hood, this is allocating a third string object and the associated memory and doing some memory copying. One way might be to malloc() on strlen(string1) + strlen(string2) + 1 char for the null terminator, then strcpy() and strcat() in the parts. There are quite a few ways this could be done, but this is one.
Now the problem lies here: in an opaque, higher-order implementation of string concatenation (for example), the best outcome we could hope for is the one which is fastest in C, ie which we discovered by trying all paths including strcat, memcpy, strcpy, etc. So let's assume that the best path was chosen for std::string's + operator overload. That still makes it only as fast as the best implementation in C. Take a step back and realise that the + operator may also do clever things like allocate more memory than required to save on a realloc() later as well as rudimentary bounds-checking or what-have-you, and it's easy to see why the string variant is 10-20x slower (https://github.com/fluffynuts/cpp_vs_c.git -- oddly enough, the disparity was greater on a win32 version I did earlier today where the sprintf() version took only 48ms for 32768 iterations vs over 600ms for the string version. The code linked here reports the following results on my Linux machine:
C++ function: 32768 runs took 5827 ms
C function: 32768 runs took 706 ms
C function (sprintf): 32768 runs took 4826 ms
Which still shows that the C++ version is nearly 10x slower. Some of the difference I've experienced between platforms may be to Microsoft optimisations as well as stdlib being (in my experience) much slower on Windows than Linux (not counting boost, which I haven't used, but which benchmarks well). The C++ version may optimise for frequent use better than the quoted C version -- but the C programmer is free to update her code accordingly, as required. Again, remember, we're talking about proficient code solutions, so when you change the parameters of the argument, the code is free to change too.
This shouldn't be surprising. The C++ code has to deal with the generic case (and provides a lot of extra functionality which is probably worth the cost) than the C version. But let me re-iterate:
Any proficient code in a higher-order language can only hope to be (at best) as proficient as proficient code in a language of lower-order for solving the same problem.
Note the use of the word "proficient". If I write shitty C code, I bet you can write good C++ that is faster. Same goes for any other pairing: if I write shitty low-order code, I'm sure you can write good high-order code which out-performs my shitty code.
The request I have to the programming community is this:
Please be honest and stop trying to win a "my language is better than yours" with pure lies and FUD. When speaking of well-written, proficient code, Ruby/Python/.NET IL/PHP/C++/whatever is NOT faster than C or assembly. The lower you go, the more specific your instructions can be and the more efficient the overall run can be.
We all accept the costs of higher-order languages because of what they offer us:
Quicker to code solutions to the problems which are interesting and/or lucrative, therefore cheaper (overall) than the iron required to run the output
Safe memory handling
Rich libraries
Abstraction from the iron
Easier-to-read (and therefore maintain) code
And a host of other benefits
I'm open for challenges on this though. Provide a problem which is relatively small to solve in a lower-order language which you think is faster (not smaller or more elegant) in a higher-order language and I'll see what I can do prove my point. Remember: a small problem, like something mathematical or string manipulation.
