Preliminary. ThinkMeta ConcurrentTasks and this documentation are preliminary; everything is subject to change.

Two schedulers

On this page

A server takes requests and runs a long computation for each. If both share one scheduler, a new request waits whenever every thread is busy computing. A higher priority does not help: a running task keeps its thread until it waits or ends. This sample gives the requests a scheduler of their own – one thread that only takes requests – and runs the computations on a batch scheduler with two threads. A request hands its computation to the batch scheduler and waits for it; meanwhile the request thread takes the next request.

The tasks

// A long computation: keeps its thread busy for 50 ms.
class Computation final : public TC::Tasks::WaitableTask_s
{
public:
    void Run() noexcept override
    {
        const auto end = Clock::now() + 50ms;
        while (Clock::now() < end) {
            // ... compute ...
        }
    }
};

// A request: runs on the request scheduler and hands its computation to the
// batch scheduler.
class Request final : public TC::Tasks::Task
{
public:
    Request(int number, TC::ITaskScheduler& batchScheduler) noexcept
        : number_(number),
          batchScheduler_(batchScheduler)
    {}

    void Run() noexcept override
    {
        std::printf("%4lld ms  request %d started\n", MillisecondsSinceStart(),
                    number_);

        // While this task waits, the request thread takes the next request.
        Computation computation;
        computation.Schedule(batchScheduler_);
        computation.Wait();

        // Back on the request scheduler.
        std::printf("%4lld ms  request %d done\n", MillisecondsSinceStart(),
                    number_);
    }

private:
    int                 number_;
    TC::ITaskScheduler& batchScheduler_;
};

Computation is a WaitableTask_s: the request that starts it is its only waiter. Schedule with a scheduler adds the task to that scheduler instead of the calling task's own. Wait suspends the request until the computation has exited; then the request goes on on the request scheduler. The computation can be a local variable: with the single-waiter variant the waiter may destroy it as soon as Wait has returned.

Two schedulers

int main()
{
    // ...
    // One thread for requests, two threads for computations.
    TC::SchedulerInfo   requestInfo{.numberOfThreads = 1};
    TC::SchedulerInfo   batchInfo{.numberOfThreads = 2};
    TC::ITaskScheduler* requestScheduler
        = TC::CreateTaskScheduler(TC::SchedulerKind::Default, &requestInfo);
    TC::ITaskScheduler* batchScheduler
        = TC::CreateTaskScheduler(TC::SchedulerKind::Default, &batchInfo);
    if (requestScheduler == nullptr || batchScheduler == nullptr)
        return 1;
    requestScheduler->Start();
    batchScheduler->Start();
    // ...
}

Each CreateTaskScheduler creates a scheduler with its own threads (numberOfThreads) and its own queues. The tasks of one scheduler never run on the threads of the other.

Sending requests

int main()
{
    // ...
    Request requests[] = {{1, *batchScheduler},
                          {2, *batchScheduler},
                          {3, *batchScheduler},
                          {4, *batchScheduler}};

    TC::Synchronization::TaskGroup_s group; // main is its only waiter
    for (Request& request : requests)
        group.AddTask(&request);

    // A request arrives every 10 ms (main is no scheduler thread, it may
    // block).
    for (Request& request : requests) {
        requestScheduler->AddTask(&request);
        std::this_thread::sleep_for(10ms);
    }
    group.Wait();
    // ...
}

Each request gets the batch scheduler in its constructor. main counts all four into a TaskGroup_s, adds one every 10 ms to the request scheduler with AddTask and waits until all have exited.

Full program

Show the whole program
// Two schedulers: one takes requests, the other runs their long computations. A
// request hands its computation to the batch scheduler and waits for it; meanwhile
// the request scheduler is free for the next request.
#include <chrono>
#include <cstdio>
#include <thread>

#include <ThinkMeta.ConcurrentTasks/Core.h>
import ThinkMeta.ConcurrentTasks.Synchronization.TaskGroup;
import ThinkMeta.ConcurrentTasks.Tasks.Task;
import ThinkMeta.ConcurrentTasks.Tasks.WaitableTask;

namespace TC = ThinkMeta::ConcurrentTasks;
using Clock  = std::chrono::steady_clock;
using namespace std::chrono_literals;

static const Clock::time_point start = Clock::now();

static long long MillisecondsSinceStart() noexcept
{
    return std::chrono::duration_cast<std::chrono::milliseconds>(Clock::now()
                                                                 - start)
        .count();
}

// A long computation: keeps its thread busy for 50 ms.
class Computation final : public TC::Tasks::WaitableTask_s
{
public:
    void Run() noexcept override
    {
        const auto end = Clock::now() + 50ms;
        while (Clock::now() < end) {
            // ... compute ...
        }
    }
};

// A request: runs on the request scheduler and hands its computation to the
// batch scheduler.
class Request final : public TC::Tasks::Task
{
public:
    Request(int number, TC::ITaskScheduler& batchScheduler) noexcept
        : number_(number),
          batchScheduler_(batchScheduler)
    {}

    void Run() noexcept override
    {
        std::printf("%4lld ms  request %d started\n", MillisecondsSinceStart(),
                    number_);

        // While this task waits, the request thread takes the next request.
        Computation computation;
        computation.Schedule(batchScheduler_);
        computation.Wait();

        // Back on the request scheduler.
        std::printf("%4lld ms  request %d done\n", MillisecondsSinceStart(),
                    number_);
    }

private:
    int                 number_;
    TC::ITaskScheduler& batchScheduler_;
};

int main()
{
    TC::ConvertThreadToTask();

    // One thread for requests, two threads for computations.
    TC::SchedulerInfo   requestInfo{.numberOfThreads = 1};
    TC::SchedulerInfo   batchInfo{.numberOfThreads = 2};
    TC::ITaskScheduler* requestScheduler
        = TC::CreateTaskScheduler(TC::SchedulerKind::Default, &requestInfo);
    TC::ITaskScheduler* batchScheduler
        = TC::CreateTaskScheduler(TC::SchedulerKind::Default, &batchInfo);
    if (requestScheduler == nullptr || batchScheduler == nullptr)
        return 1;
    requestScheduler->Start();
    batchScheduler->Start();

    {
        Request requests[] = {{1, *batchScheduler},
                              {2, *batchScheduler},
                              {3, *batchScheduler},
                              {4, *batchScheduler}};

        TC::Synchronization::TaskGroup_s group; // main is its only waiter
        for (Request& request : requests)
            group.AddTask(&request);

        // A request arrives every 10 ms (main is no scheduler thread, it may
        // block).
        for (Request& request : requests) {
            requestScheduler->AddTask(&request);
            std::this_thread::sleep_for(10ms);
        }
        group.Wait();
    }

    requestScheduler->Stop();
    batchScheduler->Stop();
    requestScheduler->Destroy();
    batchScheduler->Destroy();
    TC::ConvertTaskToThread();
    return 0;
}

Source and project on GitHub

Build and run

A console program. With the core's props file imported and the library referenced (see Set up a project) it needs nothing else. It takes about a tenth of a second; the times may vary a little:

   0 ms  request 1 started
  10 ms  request 2 started
  20 ms  request 3 started
  30 ms  request 4 started
  50 ms  request 1 done
  60 ms  request 2 done
 100 ms  request 3 done
 110 ms  request 4 done

Requests 3 and 4 start on time although both batch threads are busy. Their computations wait in the batch scheduler's queue until a batch thread is free at 50 and 60 ms. On Windows a 10 ms sleep often takes about 15 ms, so there the requests arrive a little later.

What to notice

  • Requests start on time. The request thread never runs a computation, so a new request waits at most for the short part of another request. With one scheduler for both, requests 3 and 4 would have waited for a free thread as well – with high priority too.
  • Waiting frees the thread. While a request waits for its computation, the request thread runs the next one: at 30 ms four requests are in progress on a single thread.
  • Back on its own scheduler. A batch thread ends the computation and resumes the request, but the request goes on on the request scheduler: a suspended task always resumes on the scheduler that suspended it. Tasks of different schedulers can therefore wait for each other with the usual primitives.
  • When the batch scheduler uses every processor. Here the two schedulers have three threads, fewer than most machines have processors. If the batch scheduler gets a thread per processor, the request thread competes with the batch threads for a processor. Raise its priority in a thread callback, so that the operating system preempts a batch thread for it; see Thread callbacks.

See also