rvalue references in C++

Posted on 2017-01-10

move, rvalues, forward and C++

Ever since I heard about it, the concept of move semantics has been intriguing and appealing to me. I confess to liking new C++ things and micro-optimizations way too much... but still, the concept of "moving" memory instead of copying it in order to gain efficiency is at least worth exploring, right? Anyhow, I chose a function of mine and decided to try and make it handle move semantics properly. The function I chose is one that I use often enough that I thought it would make a difference. It is the following template¹:

template< typename T >
std::string to_string( T const& to_convert )
{
   std::stringstream strm( "" );
   strm << to_convert;
   return strm.str();
}

Now, it turns out that because it is a templated function, move semantics will have to be mixed with forwarding references and that, if you ask me, those things are gibberish if you don't understand lvalues and rvalues. This is what started my adventures into rvalue land. After going through the process, I now realize that for this particular case, it is not really useful because as far as I can tell, std::strinstream::operator<< is not overloaded on rvalue references. Still, the exercise was enlightening and I now know (or at least think I know) a lot more about rvalues. The information here is nothing new, probably incomplete and also probably full of inaccuracies when compared to the strict standard C++ language definitions (after all, this is in my own words), but I have found the information useful. I guess this is a snapshot of my current understanding. All the better if it helps anybody else. Now, on to my journey.

A new kind of references: &&

The idea behind move semantics is that when you need the value a variable contains and it is a temporary (or simply about to get out of scope), you could make the memory it owns yours instead of copying the value in your own memory. This should execute faster then the copying as it is doing less work. Of course, saving copies will not make much difference is you are copying a single int, but if you are, for instance, copying a std::vector of an image class, stealing the pointer to the images instead of copying the memory should have a large impact on runtime efficiency. For this to be possible, there has to be a mechanism to detect the temporary nature of a variable and select what to do when a variable is temporary and when a variable is not. Starting with the 2011 standard, C++ now defines rvalue references, identified by a double ampersand symbol (&&), which will only bind to temporary values.

This gives us a way to determine if a parameter is a temporary value: create an overload (or a single function) which takes an rvalue reference as its input. Since this reference parameter will only bind to a temporary value, one can assume that the parameter cannot normally be used elsewhere in the program once the function returns. When this guarantee holds, one can safely steal (or move) the internals of the parameter(s) instead of making a copy(ies), potentially making the function more efficient. This is the crux of move semantics. Of course, the devil is in the details, as I have found out.

Quick digression: value categories

Lvalues and rvalues are value categories. Those two categories have been part of C++ since its first standardization in 1998 and were carried over from C (although in C, no rvalues were directly defined and "not lvalue" was deemed sufficient². Those two categories take their name from their original, although now erroneous, definition, which was to think of them as the "left" and "right" operands of an assignment:

int a = 7; // a is an lvalue and 7 is an rvalue

With that definition, "assignability" appears as the main difference between lvalues and rvalues: if something cannot be assigned to, it is an rvalue. I honestly do not know if it is the case in C, but I know that is not the case in C++. Here is an example of an lvalue that is not assignable in C++.

const int b{ 8 }; // b is an lvalue
b = 9; // error, can't assign to const even though it is an lvalue

The const variable is still an lvalue, but one cannot assign a new value to it after construction. So although it is not assignable, it is an lvalue. It is my understanding that the more interesting property is not "assignability", but rather "addressability", i.e. the capacity to refer to a value. I can take the address of both variables defined in the previous examples and refer to it, but I cannot take the address of the literals and refer to them later in the program.

int a;
int& c = &a; // valid
int& d = &8; // invalid, can't refer to (or take address of) the literal 8

So a better rule for rvalues is the following: if I can take the address of a value or expression and reference it later in the program, the value is an lvalue; otherwise, it is an rvalue.

This is not the complete picture of value categories, since five value categories are now defined in C++ (lvalues, glvalues, rvalues, prvalues and xvalues), but it turns out to be an adequate rule of thumb for lvalue/rvalue discrimination and that is what matters most for move semantics. In this blog post on isocpp.org, Scott Meyers gives this heuristics to determine the general type (lvalue vs rvalue) of an expression or variable:

If you can take the address of an expression, the expression is an lvalue.
If the type of an expression is an lvalue reference (e.g., T& or const T&, etc.), that expression is an lvalue.
Otherwise, the expression is an rvalue.

which is a quote from the linked blog post. Again, in Scott's words:

Conceptually (and typically also in fact), rvalues correspond to temporary objects, such as those returned from functions or created through implicit type conversions. Most literal values (e.g., 10 and 5.3) are also rvalues.

I think a nice feature of this heuristic is that it helps you remember that the type of an expression is independent of whether the expression is an lvalue or an rvalue. That is, given a type T, you can have lvalues of type T as well as rvalues of type T. It’s especially important to remember this when dealing with a [function] parameter of rvalue reference type, because the parameter itself is an lvalue.

That last sentence took me a while to fully understand. It is illustrated by the following example (where the new double ampersand symbol (&&) for rvalue references is used):

void foo( MyType&& param )
{
   auto& alias = &param;  // Completely legal: param has a memory location
                          // inside the function body.
   Thus, by the heuristic
                          // above, param is an lvalue.
}

This fact that the parameter is an lvalue inside the function might seem like a simple detail, but it will turn out to be necessary to explain why std::move and std::forward are necessary later on.

Overloading on rvalue reference

So the idea, as mentioned, is to create an overload on rvalue references. There is no way of representing rvalues (almost by definition from a certain point of view), so rvalue references are the only option. The following is a simple example.

#include <iostream>

int bar()
{ return 9; }

// const lvalue reference overload
void foo( int const& param )
{ std::cout << "foo( int const& param )" << std::endl; }

// rvalue reference overload
void foo( int&& param )
{ std::cout << "foo( int&& param )" << std::endl; }

// by copy overload, do not define as it will yield
//       error: call to 'foo' is ambiguous
// void foo( int param )
// { std::cout << "foo( int param )" << std::endl; }

int main( int argc, char* argv[] )
{
   int a{ 6 };
   int& b{ a };
   const int& c{ b };
   foo( a );     // calls first foo
   foo( b );     // calls first foo
   foo( c );     // calls first foo
   foo( 7 );     // calls second foo
   foo( bar() ); // calls second foo
}

In this case, this does not buy you much, but the snippet above is compilable by any C++11 conformant compiler. This little code should output foo( int const& param ) three times and foo( int&& param ) twice. A non compilable but more realistic/useful example would be:

std::vector<int> foo();

// two functions constructing a MyCoolClass object from a vector<int>
MyCoolClass make_from_std_vec( std::vector<int> const& vec ); /* overload 1 */
MyCoolClass make_from_std_vec( std::vector<int>&& vec );      /* overload 2 */

std::vector<int> the_vec;
/* fill the_vec */

auto obj_1 = make_from_std_vec( the_vec );  // binds to overload 1
auto obj_2 = make_from_std_vec( foo() );    // binds to overload 2

Here, because std::vector has been updated to allow move semantics, the second overload will be able to move the memory instead of copying it into the MyCoolClass object which should save a copy.

Implementing : enters std::move

Once you have an overload which selects the rvalues, you have to implement it. Most (if not all) POD types and STL types have been updated for move semantics, although I am not sure that moving is faster than copying in the case of int, for instance. That being said, this means that move constructors and move assignment operators are available for std::vector. If you pass an rvalue to those, they will move. But that brings us back to the discussion on function parameters inside of the function being lvalues. That means that the following will not actually move anything and will instead make a copy:

std::vector<int> steal_guts_and_do_stuff( std::vector<int>&& vec )
{
   std::vector<int> result( vec );   // INCORRECT, will not move
   /* do stuff */
   return result;
}

That is because even though there is an overload of the std::vector constructor for rvalues, what you actually passed as a parameter (vec) is not an rvalue (that was explained in previous sections). Thus, the compiler will select the copy constructor instead of the move constructor. That being said, because you are implementing the rvalue reference overload of your function (in this case, the steal_guts_and_do_stuff function), you know that in the caller scope, the parameter is actually an rvalue. This means if you had a way to cast the parameter to an rvalue inside the function implementation to tell the compiler to select the move constructor for std::vector, then that one would be selected. This can be done with the new function std::move. All this function does is unconditionally cast its input to an rvalue reference. The casting is done via the reference collapsing rules. Thus, the implementation above should actually be written as:

std::vector<int> steal_guts_and_do_stuff( std::vector<int>&& vec )
{
   std::vector<int> result( std::move( vec ) );
   /* do stuff */
   return result;
}

where you can see that std::move has been used on vec. That is pretty much it. Just insure that in your rvalue overload implementations, you use std::move on all the parameters that you want to move from and that you do not reuse those parameters after they have been moved from. If you want your user defined types to be "movable from", then define a move constructor and a move assignment operator and then a user will be able to move from your types. And notice that you do not use std::move on the return statement. Moving is done when you use the input parameter to cast it to an rvalue forcing the compiler to take the rvalue overload of the function you are calling (in my case, move constructor). You do not want to move the return.

And then they were three: T&&

As I have previously mentioned, the function that I wanted to convert to move semantics was a template. There is a catch in this case. Actually, this function declaration:

template< typename T >
void foo( T&& param );

does not declare an rvalue reference overload. The reference here is a forwarding reference³. Referencing once again Scott Meyer's blog post on isocpp.org, one finds this rule of thumb to determine the if an expression is a forwarding reference:

If a variable or parameter is declared to have type T&& for some deduced type T, that variable or parameter is a universal reference.

Deduced type mostly means templates and auto (which I won't talk about anymore). What is particular about forwarding references is that they can bind to both lvalues and rvalues, depending on what type (not value) it is initialized with. In fact, they will bind to lvalues, rvalues, const, non-const, volatile, non-volatile... This makes them really greedy and has some interesting consequences on the overload set, but that's not the subject here.

So now, three types of references have been mentioned: lvalue references, rvalue references and forwarding references. When implementing a function overload for one of the reference type, inside the function body, one must make sure to understand what should happen. For an lvalue reference overload, the parameter is an lvalue and should never be moved from. For an rvalue reference overload, the parameter is always an rvalue, thus temporary, and can always be moved from. For a forwarding reference, the parameter can be either an lvalue or an rvalue and should be moved from in the latter case, but not the former. The need to cast in the function body has been previously explained. I also mentioned that the tool to cast unconditionally to an rvalue reference is std::move. For forwarding references, the tool to conditionally cast to rvalue references is std::forward. This standard library template will cast an lvalue reference (or something that binds to an lvalue reference) to an lvalue reference and an rvalue reference (or something that binds to an rvalue reference) to an rvalue reference. The mechanism used for this is reference collapsing and the result is exactly what we need to implement the forwarding reference overload. The following code snippet illustrates what usually should be done:

MyClass from_vec( std::vector const& vec )
{
   return MyClass{ vec };
}

MyClass from_vec( std::vector&& vec )
{
   return MyClass{ std::move( vec ) };
}

template< typename C >
MyClass from_container( C&& container )
{
   return MyClass{ std::forward< C >( vec ) };
}

It should be noted that for std::forward, template argument deduction would not produce the desired results, so the type has to be repeated in the call to the function.

The result

So, now, I have everything I need to write my function. It is a template, so it will use the forwarding references. This is my new implementation:

template< typename T >
std::string to_string( T&& to_convert )
{
   std::stringstream strm( "" );
   strm << std::forward<T>( to_convert );
   return strm.str();
}

If (and as I pointed out in the intro, it is not the case) the stringstream redirect operator (<<) has an rvalue overload which steals the internals of it's argument, it will do so for the cases where to_string is called with an rvalue and it will use the regular lvalue reference overload when called with an lvalue. This is the "optimal" or near optimal behavior for my function.

To get to my implementation, I had first simply stuck a && symbol to my reference (removing the const, of course) and used std::move. It failed for reasons that are now apparent from the discussion in the previous sections. After that, I started to read all of the references I link in these notes (and more) and realized I needed to use std::forward. I finally realized that for my function, this is useless, but the journey was worth it!

Notes

^[1] Yes, I am sure someone else would do a better job at writing it more generic, faster, better, etc.; not the point! ↩︎

^[2] The C language itself borrowed the concepts from CPL (see this blog post by Danny Kalev). ↩︎

^[3] Scott Meyers used the terminology Universal References both in his articles and his book Effective Modern C++, but after discussion with members of the ISO C++ committee and the C++ community, he agreed to include a footnote (in Item 24) to say that since they should almost always be used with std::forward, the name forwarding references is gaining traction. ↩︎

^[4] Unreferenced in the text: ambiguous call