c++ coroutines
TRANSCRIPT
Your systems. Working as one.
Sumant Tambe, PhDPrincipal Research Engineer and Microsoft VC++ MVPReal-Time Innovations, Inc.@sutambe
SFBay Association of C/C++ UsersJan 13, 2016
Polyglot Programming DCMar 16, 2016
7/9/2017 © 2015 RTI 2
Alternative Title
Why should you care?
• Most software written today relies on networking and I/O
• Simplify writing I/O-oriented software– Correct (bug-free)
– Performs well
• Non-blocking
• Latency-aware
• Multi-core scalable
– Expressive–Simple and direct expression of intent
– Easy to write and read (maintainable)
– Productivity
• Modular
• Reusable
• Extensible
– and have fun while doing that!
7/9/2017 © 2015 RTI 4
Agenda
• Remote Procedure call (RPC) over DDS (DDS-RPC) standard
• Deep dive into asynchronous programming
– future<T> examples
– C++ await examples
– Abusing C++ await to avoid if-then-else boilerplate
– C++ generator examples
• Composing abstractions
7/9/2017 © 2015 RTI 5
7/9/2017 © Real-Time Innovations, Inc. 6
Data Connectivity Standard for the Industrial IoT
DDS: Data Connectivity Standard for the Industrial IoT
© 2009 Real-Time Innovations, Inc.
Streaming
DataSensors Events
Real-Time
Applications
Enterprise
ApplicationsActuators
The DDS Standard Family
8
DDS v 1.4
RTPS v2.2D
DS-
SEC
UR
ITY
DD
S-X
TYP
ES
Application
UDP TCP** DTLS** TLS**
DDS-C++ DDS-JAVA* DDS-IDL-C DDS-IDL-C#
SHARED-MEMORY**IP
DD
S-W
EB
HTTP(s)
IDL
4.0
© 2015 RTI
DD
S-R
PC
*
DDS-RPC
• Remote Procedure Call over DDS Pub-Sub Middleware
• Adopted OMG specification
• C++ and Java
• Two language bindings
– Request/Reply
– Function-call
• Reference Implementation
– dds-rpc-cxx RTI github
7/9/2017 © 2015 RTI 9
RobotControl IDL Interface
7/9/2017 © 2015 RTI 10
module robot {
exception TooFast {};
enum Command { START_COMMAND, STOP_COMMAND };
struct Status {
string msg;
};
@DDSService
interface RobotControl
{
void command(Command com);
float setSpeed(float speed) raises (TooFast);
float getSpeed();
void getStatus(out Status status);
};
}; // module robot
RobotControl Abstract Class
7/9/2017 © 2015 RTI 11
class RobotControl
{
public:
virtual void command_async(const robot::Command & command) = 0;
// returns old speed when successful
virtual float setSpeed_async(float speed) = 0;
virtual float getSpeed_async() = 0;
virtual robot::RobotControl_getStatus_Out getStatus_async() = 0;
virtual ~RobotControl() { }
};
Synchronous calls
7/9/2017 © 2015 RTI 12
robot::RobotControlSupport::Client
robot_client(rpc::ClientParams().domain_participant(...)
.service_name("RobotControl"));
float speed = 0;
try
{
speed = robot_client.getSpeed();
speed += 10;
robot_client.setSpeed(speed);
}
catch (robot::TooFast &)
{
printf("Going too fast!\n");
}
How well do you know latency?
7/9/2017 © 2015 RTI 13
Action Latency
Execute a typical instruction 1 second
Fetch from L1 cache memory 0.5 second
Branch misprediction 5 seconds
Fetch from L2 cache memory 7 seconds
Mutex lock/unlock 30 seconds
Fetch from main memory 1.5 minutes
Send 2K bytes over 1Gbps network 5.5 hours
Read 1 MB sequentially from memory 3 days
Fetch from new disk location (seek) 13 weeks
Read 1MB sequentially from disk 6.5 months
Send packet from US to Europe and back 5 years
Credit: http://www.coursera.org/course/reactive week 3-2Latency Numbers Every Programmer Should Know (jboner) https://gist.github.com/jboner/2841832
Assume a typical instruction takes 1 second…
Making Latency Explicit … as an Effect
7/9/2017 © 2015 RTI 14
class RobotControlAsync
{
public:
virtual rpc::future<void> command_async(const robot::Command & command) = 0;
// returns old speed when successful
virtual rpc::future<float> setSpeed_async(float speed) = 0;
virtual rpc::future<float> getSpeed_async() = 0;
virtual rpc::future<robot::RobotControl_getStatus_Out> getStatus_async() = 0;
virtual ~RobotControlAsync() { }
};
When rpc::future is C++11 std::future
7/9/2017 © 2015 RTI 15
try {
dds::rpc::future<float> speed_fut =
robot_client.getSpeed_async();
// Do some other stuff
while(speed_fut.wait_for(std::chrono::seconds(1)) ==
std::future_status::timeout);
speed = speed_fut.get();
speed += 10;
dds::rpc::future<float> set_speed_fut =
robot_client.setSpeed_async(speed);
// Do even more stuff
while(set_speed_fut.wait_for(std::chrono::seconds(1)) ==
std::future_status::timeout);
set_speed_fut.get();
}
catch (robot::TooFast &) {
printf("Going too fast!\n");
}
Limitations of C++11 std::future<T>
• Must block (in most cases) to retrieve the result
• If the main program isn’t blocked, it’s likely that the continuation is blocked– I.e., the async result is available but no one has
noticed
• The programmer must do correlation of requests with responses– The order in which async result will be ready is not
guaranteed by DDS-RPC (when multiple requests are outstanding)
7/9/2017 © 2015 RTI 16
When rpc::future is C++11 std::future
7/9/2017 © 2015 RTI 17
Composable Futures to Rescue
• Concurrency TS/C++17
• Serial Composition– future.then()
• Parallel composition– when_all, when_any
• Lot of implementations
– Boost.future, Microsoft PPL, HPX, Facebook’s Folly
– dds::rpc::future<T>wraps PPL (code)
7/9/2017 © 2015 RTI 18
Using future.then()
7/9/2017 © 2015 RTI 19
robot_client
.getSpeed_async()
.then([robot_client](future<float> && speed_fut) {
float speed = speed_fut.get();
printf("getSpeed = %f\n", speed);
speed += 10;
return robot_client.setSpeed_async(speed);
})
.then([](future<float> && speed_fut) {
try {
float speed = speed_fut.get();
printf("speed set successfully.\n");
}
catch (robot::TooFast &) {
printf("Going too fast!\n");
}
});
Improvements over C++11 future
• Main thread does not have to block
• Callback (continuation) does not have to block– The thread setting the future value invokes the
callback right away
• Request/Reply correlation isn’t explicit because the callback lambda captures the necessary state– No incidental data structures necessary (state
machines, std::map) for request/reply correlation (see Sean Parent’s CppCon’15 talk about no incidental data structures)
• Same pattern in Javascript promises and other places
7/9/2017 © 2015 RTI 20
7/9/2017 © 2015 RTI 21
Speed up the robot to MAX_SPEEDin increments of 10 and without
blocking
7/9/2017 © 2015 RTI 22
dds::rpc::future<float> speedup_until_maxspeed(
robot::RobotControlSupport::Client & robot_client)
{
static const int increment = 10;
return
robot_client
.getSpeed_async()
.then([robot_client](future<float> && speed_fut) {
float speed = speed_fut.get();
speed += increment;
if(speed <= MAX_SPEED) {
printf("speedup_until_maxspeed: new speed = %f\n", speed);
return robot_client.setSpeed_async(speed);
}
else
return dds::rpc::details::make_ready_future(speed);
})
.then([robot_client](future<float> && speed_fut) {
float speed = speed_fut.get();
if(speed + increment <= MAX_SPEED)
return speedup_until_maxspeed(robot_client);
else
return dds::rpc::details::make_ready_future(speed);
});
}
Return ready future? Why not speed?
Is that recursive? Does the stack grow?... No!
What are these lambdas doing here?Is that a CPS transform? … Yes!
.then() in Action
7/9/2017 © 2015 RTI 23
Setup getSpeed Callback Receive getSpeed Reply and invoke setSpeed
Setup setSpeed Callback Receive setSpeed Reply and invoke speedup_until_maxspeed
.then .then .then .then .then
speedup_until_maxspeed speedup_until_maxspeed speedup_until_maxspeed
What’s Wrong with .then()
• Control-flow is awkward
– The last example shows that async looping is hard
• Debugging is hard
– No stack-trace (at least not very useful)
– See screenshots
– .then is stitching together lambdas (program fragments). Not seamless.
– Awkward physical and temporal continuity
7/9/2017 © 2015 RTI 24
Welcome C++ Coroutines
7/9/2017 © 2015 RTI 25
Using C++ await
7/9/2017 © 2015 RTI 26
dds::rpc::future<void> test_iterative_await(
robot::RobotControlSupport::Client & robot_client)
{
static const int inc = 10;
float speed = 0;
while ((speed = await robot_client.getSpeed_async())+inc <= MAX_SPEED)
{
await robot_client.setSpeed_async(speed + inc);
printf("current speed = %f\n", speed + inc);
}
}
Synchronous-looking but completely non-blocking code
C++ await Examples in Visual Studio 2015
• test_await
• test_iterative_await
• test_three_getspeed_await
• test_three_setspeed_await
• test_lambda_await
• test_foreach_await
• test_accumulate_await
• test_lift_accumulate_await
code on github (robot_func.cxx)
7/9/2017 © 2015 RTI 27
Awaitable Types
• Many, many possibilities. You define the semantics.• A pair of types
– Promise and Future
• Awaitable Implements– await_ready– await_suspend– await_resume– type::promise_type
• Promise Implements– get_return_object– initial_suspend– final_suspend– set_exception– return_value
7/9/2017 © 2015 RTI 28
C++ Generators
7/9/2017 © 2015 RTI 32
• Synchronous generator (yield) is syntax sugar for creating lazy containers
• std::experimental::generator<T>
– movable
• Many more possibilities
C++ Generators
7/9/2017 © 2015 RTI 33
• Synchronous generator (yield) is syntax sugar for creating lazy containers
void test_hello(){for (auto ch: hello()){std::cout << ch;
}}
std::experimental::generator<char> hello(){yield 'H';yield 'e';yield 'l';yield 'l';yield 'o';yield ',';yield ' ';yield 'w';yield 'o';yield 'r';yield 'l';yield 'd';
}
Generator Iterator Category
• What is it?
7/9/2017 © 2015 RTI 34
std::input_iterator_tag
Deterministic Resource Cleanup
7/9/2017 © 2015 RTI 35
std::experimental::generator<std::string> read_file(const std::string & filename)
{std::fstream in(filename);std::string str;while (in >> str)yield str;
}
void test_read_file(const std::string filename){{auto & generator = read_file(filename);for (auto & str : generator){std::cout << str << " ";break;
}std::cout << "\n";
}// file closed before reaching here
}
C++ Generators
7/9/2017 © 2015 RTI 36
• Generators preserve the local variable and arguments
void test_range(){for (auto i: range(1, 10)){std::cout << i;
}}
std::experimental::generator<int> range(int start, int count){if (count > 0) {for (int i = 0; i < count; i++)yield start + i;
}}
Recursive?
7/9/2017 © 2015 RTI 37
std::experimental::generator<int> range(int start, int count){if (count > 0) {yield start;for (auto i : range(start + 1, count - 1))yield i;
}}
• Crashes at count ~3000– Quadratic run-time complexity
– i-th value yields i times
• “Regular” generators can’t suspend nested stack frames– Use recursive_generator<T>
Composing Abstractions
• A cool way to compose generators is….
7/9/2017 © 2015 RTI 38
range-v3
• But range-v3 did not compile on VS2015– So I’ll use my own generators library– See my Silicon Valley Code Camp 2015 talk
• Composable Generators and Property-based Testing (video, slides)
– You can use boost.range if you like
• A cool way to compose lazy containers is….
Composing Generators
7/9/2017 © 2015 RTI 39
#include “generator.h” // See my github
std::experimental::generator<char> hello(){for (auto ch : "HELLO WORLD")yield ch;
}
void test_coroutine_gen(){std::string msg = “hello@world";
auto gen = gen::make_coroutine_gen(hello).map([](char ch) { return char(ch + 32); }).take(msg.size());
int i = 0;
for (auto ch: gen){std::cout << ch;assert(ch == msg[i++]);
}}
Generator and Synchrony are Orthogonal
• Yellow Boxes
– Libraries already exist (e.g., range-v3, future::then, RxCpp)
• Not really easy
– The C++ Resumable Functions proposal adds language-level support
• Like many other languages: Javascript, Dart, Hack, C#, etc.
7/9/2017 © 2015 RTI 40
Sync/Async, One/Many
Single Multiple
Sync T generator<T>::iterator
Async future<T> async_generator<T>
See Spicing Up Dart with Side Effects - ACM Queue--- Erik Meijer, Applied Duality; Kevin Millikin, Google; Gilad Bracha, Google
Further Reading
• Resumable Functions in C++: http://blogs.msdn.com/b/vcblog/archive/2014/11/12/resumable-functions-in-c.aspx
• Resumable Functions (revision 4)---Gor Nishanov, Jim Radigan (Microsoft) N4402
• Spicing Up Dart with Side Effects - ACM Queue --- Erik Meijer, Applied Duality; Kevin Millikin, Google; Gilad Bracha, Google
7/9/2017 © 2015 RTI 41