Two schedulers
A server takes requests and runs a long computation for each. If both share one scheduler, a new request waits whenever every thread is busy computing. A higher priority does not help: a running task keeps its thread until it waits or ends. This sample gives the requests a scheduler of their own – one thread that only takes requests – and runs the computations on a batch scheduler with two threads. A request hands its computation to the batch scheduler and waits for it; meanwhile the request thread takes the next request.
The tasks
// A long computation: keeps its thread busy for 50 ms.
class Computation final : public TC::Tasks::WaitableTask_s
{
public:
void Run() noexcept override
{
const auto end = Clock::now() + 50ms;
while (Clock::now() < end) {
// ... compute ...
}
}
};
// A request: runs on the request scheduler and hands its computation to the
// batch scheduler.
class Request final : public TC::Tasks::Task
{
public:
Request(int number, TC::ITaskScheduler& batchScheduler) noexcept
: number_(number),
batchScheduler_(batchScheduler)
{}
void Run() noexcept override
{
std::printf("%4lld ms request %d started\n", MillisecondsSinceStart(),
number_);
// While this task waits, the request thread takes the next request.
Computation computation;
computation.Schedule(batchScheduler_);
computation.Wait();
// Back on the request scheduler.
std::printf("%4lld ms request %d done\n", MillisecondsSinceStart(),
number_);
}
private:
int number_;
TC::ITaskScheduler& batchScheduler_;
};Computation is a WaitableTask_s: the request that starts it is its only
waiter. Schedule with a scheduler adds the task to that scheduler instead of
the calling task's own. Wait suspends the request until the computation
has exited; then the request goes on on the request scheduler. The computation can be a local
variable: with the single-waiter variant the waiter may destroy it as soon as Wait has returned.
Two schedulers
int main()
{
// ...
// One thread for requests, two threads for computations.
TC::SchedulerInfo requestInfo{.numberOfThreads = 1};
TC::SchedulerInfo batchInfo{.numberOfThreads = 2};
TC::ITaskScheduler* requestScheduler
= TC::CreateTaskScheduler(TC::SchedulerKind::Default, &requestInfo);
TC::ITaskScheduler* batchScheduler
= TC::CreateTaskScheduler(TC::SchedulerKind::Default, &batchInfo);
if (requestScheduler == nullptr || batchScheduler == nullptr)
return 1;
requestScheduler->Start();
batchScheduler->Start();
// ...
}Each CreateTaskScheduler creates a scheduler with its own threads (numberOfThreads) and its own queues. The tasks of one scheduler never run on the threads of the other.
Sending requests
int main()
{
// ...
Request requests[] = {{1, *batchScheduler},
{2, *batchScheduler},
{3, *batchScheduler},
{4, *batchScheduler}};
TC::Synchronization::TaskGroup_s group; // main is its only waiter
for (Request& request : requests)
group.AddTask(&request);
// A request arrives every 10 ms (main is no scheduler thread, it may
// block).
for (Request& request : requests) {
requestScheduler->AddTask(&request);
std::this_thread::sleep_for(10ms);
}
group.Wait();
// ...
}Each request gets the batch scheduler in its constructor. main counts all four into a
TaskGroup_s, adds one every 10 ms to the request scheduler with
AddTask and waits until all have exited.
Full program
Show the whole program
// Two schedulers: one takes requests, the other runs their long computations. A
// request hands its computation to the batch scheduler and waits for it; meanwhile
// the request scheduler is free for the next request.
#include <chrono>
#include <cstdio>
#include <thread>
#include <ThinkMeta.ConcurrentTasks/Core.h>
import ThinkMeta.ConcurrentTasks.Synchronization.TaskGroup;
import ThinkMeta.ConcurrentTasks.Tasks.Task;
import ThinkMeta.ConcurrentTasks.Tasks.WaitableTask;
namespace TC = ThinkMeta::ConcurrentTasks;
using Clock = std::chrono::steady_clock;
using namespace std::chrono_literals;
static const Clock::time_point start = Clock::now();
static long long MillisecondsSinceStart() noexcept
{
return std::chrono::duration_cast<std::chrono::milliseconds>(Clock::now()
- start)
.count();
}
// A long computation: keeps its thread busy for 50 ms.
class Computation final : public TC::Tasks::WaitableTask_s
{
public:
void Run() noexcept override
{
const auto end = Clock::now() + 50ms;
while (Clock::now() < end) {
// ... compute ...
}
}
};
// A request: runs on the request scheduler and hands its computation to the
// batch scheduler.
class Request final : public TC::Tasks::Task
{
public:
Request(int number, TC::ITaskScheduler& batchScheduler) noexcept
: number_(number),
batchScheduler_(batchScheduler)
{}
void Run() noexcept override
{
std::printf("%4lld ms request %d started\n", MillisecondsSinceStart(),
number_);
// While this task waits, the request thread takes the next request.
Computation computation;
computation.Schedule(batchScheduler_);
computation.Wait();
// Back on the request scheduler.
std::printf("%4lld ms request %d done\n", MillisecondsSinceStart(),
number_);
}
private:
int number_;
TC::ITaskScheduler& batchScheduler_;
};
int main()
{
TC::ConvertThreadToTask();
// One thread for requests, two threads for computations.
TC::SchedulerInfo requestInfo{.numberOfThreads = 1};
TC::SchedulerInfo batchInfo{.numberOfThreads = 2};
TC::ITaskScheduler* requestScheduler
= TC::CreateTaskScheduler(TC::SchedulerKind::Default, &requestInfo);
TC::ITaskScheduler* batchScheduler
= TC::CreateTaskScheduler(TC::SchedulerKind::Default, &batchInfo);
if (requestScheduler == nullptr || batchScheduler == nullptr)
return 1;
requestScheduler->Start();
batchScheduler->Start();
{
Request requests[] = {{1, *batchScheduler},
{2, *batchScheduler},
{3, *batchScheduler},
{4, *batchScheduler}};
TC::Synchronization::TaskGroup_s group; // main is its only waiter
for (Request& request : requests)
group.AddTask(&request);
// A request arrives every 10 ms (main is no scheduler thread, it may
// block).
for (Request& request : requests) {
requestScheduler->AddTask(&request);
std::this_thread::sleep_for(10ms);
}
group.Wait();
}
requestScheduler->Stop();
batchScheduler->Stop();
requestScheduler->Destroy();
batchScheduler->Destroy();
TC::ConvertTaskToThread();
return 0;
}Build and run
A console program. With the core's props file imported and the library referenced (see Set up a project) it needs nothing else. It takes about a tenth of a second; the times may vary a little:
0 ms request 1 started
10 ms request 2 started
20 ms request 3 started
30 ms request 4 started
50 ms request 1 done
60 ms request 2 done
100 ms request 3 done
110 ms request 4 doneRequests 3 and 4 start on time although both batch threads are busy. Their computations wait in the batch scheduler's queue until a batch thread is free at 50 and 60 ms. On Windows a 10 ms sleep often takes about 15 ms, so there the requests arrive a little later.
What to notice
- Requests start on time. The request thread never runs a computation, so a new request waits at most for the short part of another request. With one scheduler for both, requests 3 and 4 would have waited for a free thread as well – with high priority too.
- Waiting frees the thread. While a request waits for its computation, the request thread runs the next one: at 30 ms four requests are in progress on a single thread.
- Back on its own scheduler. A batch thread ends the computation and resumes the request, but the request goes on on the request scheduler: a suspended task always resumes on the scheduler that suspended it. Tasks of different schedulers can therefore wait for each other with the usual primitives.
- When the batch scheduler uses every processor. Here the two schedulers have three threads, fewer than most machines have processors. If the batch scheduler gets a thread per processor, the request thread competes with the batch threads for a processor. Raise its priority in a thread callback, so that the operating system preempts a batch thread for it; see Thread callbacks.