OpenSDN source code
TaskScheduler Class Reference

The TaskScheduler keeps track of what tasks are currently schedulable. When a task is enqueued it is added to the run queue or the pending queue depending as to whether there is a runable or pending task ahead of it that violates the mutual exclusion policies. When tasks exit the scheduler re-examines the tasks on the pending queue which may now be runnable. It is important that this process is efficient such that exit events do not scan tasks that are not waiting on a particular task id or task instance to have a 0 count. More...

#include <task.h>

Collaboration diagram for TaskScheduler:

Public Types

enum  CancelReturnCode { CANCELLED , FAILED , QUEUED }
 
typedef boost::function< void(const char *file_name, uint32_t line_no, const Task *task, const char *description, uint64_t delay)> LogFn
 

Public Member Functions

 TaskScheduler (int thread_count=0)
 TaskScheduler constructor. TBB assumes it can use the "thread" invoking tbb::scheduler can be used for task scheduling. But, in our case we dont want "main" thread to be part of tbb. So, initialize TBB with one thread more than its default. More...
 
 ~TaskScheduler ()
 Frees up the task_entry_db_ allocated for scheduler. More...
 
void Enqueue (Task *task)
 Enqueues a task for running. Starts task if all policy rules are met else puts task in waitq. Enqueueing may may result in the task being immedietly added to the run queue or to a pending queue. Tasks may not be added to the run queue in violation of their exclusion policy. More...
 
void EnqueueUnLocked (Task *task)
 
CancelReturnCode Cancel (Task *task)
 Cancels a Task that can be in RUN/WAIT state. The caller needs to ensure that the task exists when Cancel() is invoked. More...
 
void SetPolicy (int task_id, TaskPolicy &policy)
 Sets the task exclusion policy. Adds policy entries for the task Examples: More...
 
bool GetRunStatus ()
 
int GetTaskId (const std::string &name)
 
std::string GetTaskName (int task_id) const
 
TaskStatsGetTaskGroupStats (int task_id)
 
TaskStatsGetTaskStats (int task_id)
 
TaskStatsGetTaskStats (int task_id, int instance_id)
 
void ClearTaskGroupStats (int task_id)
 
void ClearTaskStats (int task_id)
 
void ClearTaskStats (int task_id, int instance_id)
 
TaskGroupGetTaskGroup (int task_id)
 Get TaskGroup for a task_id. Grows task_entry_db_ if necessary. More...
 
TaskGroupQueryTaskGroup (int task_id)
 Query TaskGroup for a task_id.Assumes valid entry is present for task_id. More...
 
bool IsTaskGroupEmpty (int task_id) const
 Check if there are any Tasks in the given TaskGroup. Assumes that all task ids are mutually exclusive with bgp::Config. More...
 
TaskEntryGetTaskEntry (int task_id, int instance_id)
 Get TaskGroup for a task_id. Grows task_entry_db_ if necessary. More...
 
TaskEntryQueryTaskEntry (int task_id, int instance_id)
 Query TaskEntry for a task-id and task-instance. More...
 
void OnTaskExit (Task *task)
 Method invoked on exit of a Task. Exit of a task can potentially start tasks in pendingq. More...
 
void Stop ()
 Stops scheduling of all tasks. More...
 
void Start ()
 Starts scheduling of all tasks. More...
 
void Print ()
 Debug print routine. More...
 
bool IsEmpty (bool running_only=false)
 Returns true if there are no tasks running and/or enqueued If running_only is true, enqueued tasks are ignored i.e. return true if there are no running tasks. Ignore TaskGroup or TaskEntry if it is disabled. More...
 
void Terminate ()
 
int HardwareThreadCount ()
 
uint64_t enqueue_count () const
 
uint64_t done_count () const
 
uint64_t cancel_count () const
 
void SetMaxThreadCount (int n)
 Force number of threads. More...
 
void GetSandeshData (SandeshTaskScheduler *resp, bool summary)
 
void Log (const char *file_name, uint32_t line_no, const Task *task, const char *description, uint64_t delay)
 
void RegisterLog (LogFn fn)
 
void SetTrackRunTime (bool value)
 
bool track_run_time () const
 
void EnableLatencyThresholds (uint32_t execute, uint32_t schedule)
 Enable logging of tasks exceeding configured latency. More...
 
uint32_t schedule_delay () const
 
uint32_t execute_delay () const
 
bool measure_delay () const
 
void SetLatencyThreshold (const std::string &name, uint32_t execute, uint32_t schedule)
 
uint32_t schedule_delay (Task *task) const
 
uint32_t execute_delay (Task *task) const
 
void set_event_manager (EventManager *evm)
 
void DisableTaskGroup (int task_id)
 
void EnableTaskGroup (int task_id)
 
void DisableTaskEntry (int task_id, int instance_id)
 
void EnableTaskEntry (int task_id, int instance_id)
 
void ModifyTbbKeepAwakeTimeout (uint32_t timeout)
 
void EnableMonitor (EventManager *evm, uint64_t tbb_keepawake_time_msec, uint64_t inactivity_time_msec, uint64_t poll_interval_msec)
 Enable Task monitoring. More...
 
const TaskMonitortask_monitor () const
 
const TaskTbbKeepAwaketbb_awake_task () const
 
bool use_spawn () const
 
oneapi::tbb::task_arena & tbb_arena ()
 returns current TBB arena. More...
 

Static Public Member Functions

static void Initialize (uint32_t thread_count=0, EventManager *evm=NULL)
 
static TaskSchedulerGetInstance ()
 
static int GetThreadCount (int thread_count=0)
 Get number of tbb worker threads. For testing purposes only. Limit the number of tbb worker threads. More...
 
static bool ShouldUseSpawn ()
 
static int GetDefaultThreadCount ()
 
static void SetThreadAmpFactor (int n)
 following function allows one to increase max num of threads used by TBB More...
 

Private Types

typedef std::vector< TaskGroup * > TaskGroupDb
 
typedef std::map< std::string, int > TaskIdMap
 

Private Member Functions

void SetRunningTask (Task *)
 This function should not be called in production code. It is only for unit testing to control current running task This function modifies the running task as specified by the input. More...
 
void ClearRunningTask ()
 
 DISALLOW_COPY_AND_ASSIGN (TaskScheduler)
 

Private Attributes

bool use_spawn_
 Use spawn() to run a tbb::task instead of enqueue() More...
 
TaskEntrystop_entry_
 
oneapi::tbb::global_control tbb_global_control_
 
oneapi::tbb::task_arena task_scheduler_
 
std::mutex mutex_
 
bool running_
 
uint64_t seqno_
 
TaskGroupDb task_group_db_
 
std::shared_mutex id_map_mutex_
 
TaskIdMap id_map_
 
int id_max_
 
LogFn log_fn_
 
int hw_thread_count_
 
bool track_run_time_
 
bool measure_delay_
 
uint32_t schedule_delay_
 Log if time between enqueue and task-execute exceeds the delay. More...
 
uint32_t execute_delay_
 Log if time taken to execute exceeds the delay. More...
 
uint64_t enqueue_count_
 
uint64_t done_count_
 
uint64_t cancel_count_
 
EventManagerevm_
 
TaskTbbKeepAwaketbb_awake_task_
 
TaskMonitortask_monitor_
 

Static Private Attributes

static const int kVectorGrowSize = 16
 
static boost::scoped_ptr< TaskSchedulersingleton_
 
static int ThreadAmpFactor_ = 1
 following variable allows one to increase max num of threads used by TBB More...
 

Friends

class ConcurrencyScope
 

Detailed Description

The TaskScheduler keeps track of what tasks are currently schedulable. When a task is enqueued it is added to the run queue or the pending queue depending as to whether there is a runable or pending task ahead of it that violates the mutual exclusion policies. When tasks exit the scheduler re-examines the tasks on the pending queue which may now be runnable. It is important that this process is efficient such that exit events do not scan tasks that are not waiting on a particular task id or task instance to have a 0 count.

Definition at line 306 of file task.h.

Member Typedef Documentation

◆ LogFn

typedef boost::function<void(const char *file_name, uint32_t line_no, const Task *task, const char *description, uint64_t delay)> TaskScheduler::LogFn

Definition at line 310 of file task.h.

◆ TaskGroupDb

typedef std::vector<TaskGroup *> TaskScheduler::TaskGroupDb
private

Definition at line 465 of file task.h.

◆ TaskIdMap

typedef std::map<std::string, int> TaskScheduler::TaskIdMap
private

Definition at line 466 of file task.h.

Member Enumeration Documentation

◆ CancelReturnCode

Enumerator
CANCELLED 
FAILED 
QUEUED 

Definition at line 333 of file task.h.

Constructor & Destructor Documentation

◆ TaskScheduler()

TaskScheduler::TaskScheduler ( int  thread_count = 0)

TaskScheduler constructor. TBB assumes it can use the "thread" invoking tbb::scheduler can be used for task scheduling. But, in our case we dont want "main" thread to be part of tbb. So, initialize TBB with one thread more than its default.

Definition at line 466 of file task.cc.

Here is the call graph for this function:

◆ ~TaskScheduler()

TaskScheduler::~TaskScheduler ( )

Frees up the task_entry_db_ allocated for scheduler.

Definition at line 481 of file task.cc.

Member Function Documentation

◆ Cancel()

TaskScheduler::CancelReturnCode TaskScheduler::Cancel ( Task task)

Cancels a Task that can be in RUN/WAIT state. The caller needs to ensure that the task exists when Cancel() is invoked.

Definition at line 726 of file task.cc.

Here is the call graph for this function:

◆ cancel_count()

uint64_t TaskScheduler::cancel_count ( ) const
inline

Definition at line 415 of file task.h.

◆ ClearRunningTask()

void TaskScheduler::ClearRunningTask ( )
private

Definition at line 980 of file task.cc.

◆ ClearTaskGroupStats()

void TaskScheduler::ClearTaskGroupStats ( int  task_id)

Definition at line 902 of file task.cc.

Here is the call graph for this function:

◆ ClearTaskStats() [1/2]

void TaskScheduler::ClearTaskStats ( int  task_id)

Definition at line 910 of file task.cc.

Here is the call graph for this function:

◆ ClearTaskStats() [2/2]

void TaskScheduler::ClearTaskStats ( int  task_id,
int  instance_id 
)

Definition at line 918 of file task.cc.

Here is the call graph for this function:

◆ DisableTaskEntry()

void TaskScheduler::DisableTaskEntry ( int  task_id,
int  instance_id 
)

Definition at line 1006 of file task.cc.

Here is the call graph for this function:

◆ DisableTaskGroup()

void TaskScheduler::DisableTaskGroup ( int  task_id)

Definition at line 989 of file task.cc.

Here is the call graph for this function:

◆ DISALLOW_COPY_AND_ASSIGN()

TaskScheduler::DISALLOW_COPY_AND_ASSIGN ( TaskScheduler  )
private

◆ done_count()

uint64_t TaskScheduler::done_count ( ) const
inline

Definition at line 414 of file task.h.

◆ EnableLatencyThresholds()

void TaskScheduler::EnableLatencyThresholds ( uint32_t  execute,
uint32_t  schedule 
)

Enable logging of tasks exceeding configured latency.

Definition at line 621 of file task.cc.

◆ EnableMonitor()

void TaskScheduler::EnableMonitor ( EventManager evm,
uint64_t  tbb_keepawake_time_msec,
uint64_t  inactivity_time_msec,
uint64_t  poll_interval_msec 
)

Enable Task monitoring.

Definition at line 537 of file task.cc.

Here is the call graph for this function:

◆ EnableTaskEntry()

void TaskScheduler::EnableTaskEntry ( int  task_id,
int  instance_id 
)

Definition at line 1011 of file task.cc.

Here is the call graph for this function:

◆ EnableTaskGroup()

void TaskScheduler::EnableTaskGroup ( int  task_id)

Definition at line 999 of file task.cc.

Here is the call graph for this function:

◆ Enqueue()

void TaskScheduler::Enqueue ( Task task)

Enqueues a task for running. Starts task if all policy rules are met else puts task in waitq. Enqueueing may may result in the task being immedietly added to the run queue or to a pending queue. Tasks may not be added to the run queue in violation of their exclusion policy.

Definition at line 661 of file task.cc.

Here is the call graph for this function:

◆ enqueue_count()

uint64_t TaskScheduler::enqueue_count ( ) const
inline

Definition at line 413 of file task.h.

◆ EnqueueUnLocked()

void TaskScheduler::EnqueueUnLocked ( Task task)

Definition at line 666 of file task.cc.

Here is the call graph for this function:

◆ execute_delay() [1/2]

uint32_t TaskScheduler::execute_delay ( ) const
inline

Definition at line 430 of file task.h.

◆ execute_delay() [2/2]

uint32_t TaskScheduler::execute_delay ( Task task) const

Definition at line 567 of file task.cc.

◆ GetDefaultThreadCount()

int TaskScheduler::GetDefaultThreadCount ( )
static

Definition at line 451 of file task.cc.

◆ GetInstance()

TaskScheduler * TaskScheduler::GetInstance ( )
static

Definition at line 573 of file task.cc.

Here is the call graph for this function:

◆ GetRunStatus()

bool TaskScheduler::GetRunStatus ( )
inline

Definition at line 355 of file task.h.

◆ GetSandeshData()

void TaskScheduler::GetSandeshData ( SandeshTaskScheduler *  resp,
bool  summary 
)

Definition at line 1592 of file task.cc.

Here is the call graph for this function:

◆ GetTaskEntry()

TaskEntry * TaskScheduler::GetTaskEntry ( int  task_id,
int  instance_id 
)

Get TaskGroup for a task_id. Grows task_entry_db_ if necessary.

Definition at line 609 of file task.cc.

Here is the call graph for this function:

◆ GetTaskGroup()

TaskGroup * TaskScheduler::GetTaskGroup ( int  task_id)

Get TaskGroup for a task_id. Grows task_entry_db_ if necessary.

Definition at line 580 of file task.cc.

◆ GetTaskGroupStats()

TaskStats * TaskScheduler::GetTaskGroupStats ( int  task_id)

Definition at line 926 of file task.cc.

Here is the call graph for this function:

◆ GetTaskId()

int TaskScheduler::GetTaskId ( const std::string &  name)

Definition at line 883 of file task.cc.

◆ GetTaskName()

std::string TaskScheduler::GetTaskName ( int  task_id) const

Definition at line 873 of file task.cc.

◆ GetTaskStats() [1/2]

TaskStats * TaskScheduler::GetTaskStats ( int  task_id)

Definition at line 934 of file task.cc.

Here is the call graph for this function:

◆ GetTaskStats() [2/2]

TaskStats * TaskScheduler::GetTaskStats ( int  task_id,
int  instance_id 
)

Definition at line 942 of file task.cc.

Here is the call graph for this function:

◆ GetThreadCount()

int TaskScheduler::GetThreadCount ( int  thread_count = 0)
static

Get number of tbb worker threads. For testing purposes only. Limit the number of tbb worker threads.

Definition at line 428 of file task.cc.

◆ HardwareThreadCount()

int TaskScheduler::HardwareThreadCount ( )
inline

Definition at line 404 of file task.h.

◆ Initialize()

void TaskScheduler::Initialize ( uint32_t  thread_count = 0,
EventManager evm = NULL 
)
static

Definition at line 504 of file task.cc.

Here is the call graph for this function:

◆ IsEmpty()

bool TaskScheduler::IsEmpty ( bool  running_only = false)

Returns true if there are no tasks running and/or enqueued If running_only is true, enqueued tasks are ignored i.e. return true if there are no running tasks. Ignore TaskGroup or TaskEntry if it is disabled.

Definition at line 850 of file task.cc.

Here is the call graph for this function:

◆ IsTaskGroupEmpty()

bool TaskScheduler::IsTaskGroupEmpty ( int  task_id) const

Check if there are any Tasks in the given TaskGroup. Assumes that all task ids are mutually exclusive with bgp::Config.

Definition at line 600 of file task.cc.

Here is the call graph for this function:

◆ Log()

void TaskScheduler::Log ( const char *  file_name,
uint32_t  line_no,
const Task task,
const char *  description,
uint64_t  delay 
)

Definition at line 549 of file task.cc.

◆ measure_delay()

bool TaskScheduler::measure_delay ( ) const
inline

Definition at line 432 of file task.h.

◆ ModifyTbbKeepAwakeTimeout()

void TaskScheduler::ModifyTbbKeepAwakeTimeout ( uint32_t  timeout)

Definition at line 531 of file task.cc.

Here is the call graph for this function:

◆ OnTaskExit()

void TaskScheduler::OnTaskExit ( Task task)

Method invoked on exit of a Task. Exit of a task can potentially start tasks in pendingq.

Definition at line 789 of file task.cc.

Here is the call graph for this function:

◆ Print()

void TaskScheduler::Print ( )

Debug print routine.

Definition at line 835 of file task.cc.

Here is the call graph for this function:

◆ QueryTaskEntry()

TaskEntry * TaskScheduler::QueryTaskEntry ( int  task_id,
int  instance_id 
)

Query TaskEntry for a task-id and task-instance.

Definition at line 614 of file task.cc.

Here is the call graph for this function:

◆ QueryTaskGroup()

TaskGroup * TaskScheduler::QueryTaskGroup ( int  task_id)

Query TaskGroup for a task_id.Assumes valid entry is present for task_id.

Definition at line 596 of file task.cc.

◆ RegisterLog()

void TaskScheduler::RegisterLog ( LogFn  fn)

Definition at line 557 of file task.cc.

◆ schedule_delay() [1/2]

uint32_t TaskScheduler::schedule_delay ( ) const
inline

Definition at line 429 of file task.h.

◆ schedule_delay() [2/2]

uint32_t TaskScheduler::schedule_delay ( Task task) const

Definition at line 561 of file task.cc.

◆ set_event_manager()

void TaskScheduler::set_event_manager ( EventManager evm)

Definition at line 519 of file task.cc.

Here is the call graph for this function:

◆ SetLatencyThreshold()

void TaskScheduler::SetLatencyThreshold ( const std::string &  name,
uint32_t  execute,
uint32_t  schedule 
)

Definition at line 628 of file task.cc.

Here is the call graph for this function:

◆ SetMaxThreadCount()

void TaskScheduler::SetMaxThreadCount ( int  n)

Force number of threads.

◆ SetPolicy()

void TaskScheduler::SetPolicy ( int  task_id,
TaskPolicy policy 
)

Sets the task exclusion policy. Adds policy entries for the task Examples:

  • Policy <tid0> => <tid1, -1> <tid2, inst2> will result in following:
    • task_db_[tid0] : Rule <tid1, -1> is added to policyq
    • task_group_db_[tid0, inst2] : Rule <tid2, inst2> is added to policyq
  • The symmetry of policy will result in following additional rules,
    • task_db_[tid1] : Rule <tid0, -1> is added to policyq
    • task_group_db_[tid2, inst2] : Rule <tid0, inst2> is added to policyq

Definition at line 636 of file task.cc.

Here is the call graph for this function:

◆ SetRunningTask()

void TaskScheduler::SetRunningTask ( Task unit_test)
private

This function should not be called in production code. It is only for unit testing to control current running task This function modifies the running task as specified by the input.

Definition at line 975 of file task.cc.

Here is the call graph for this function:

◆ SetThreadAmpFactor()

void TaskScheduler::SetThreadAmpFactor ( int  n)
static

following function allows one to increase max num of threads used by TBB

Definition at line 985 of file task.cc.

◆ SetTrackRunTime()

void TaskScheduler::SetTrackRunTime ( bool  value)
inline

Definition at line 424 of file task.h.

◆ ShouldUseSpawn()

bool TaskScheduler::ShouldUseSpawn ( )
static

Definition at line 455 of file task.cc.

◆ Start()

void TaskScheduler::Start ( )

Starts scheduling of all tasks.

Definition at line 825 of file task.cc.

Here is the call graph for this function:

◆ Stop()

void TaskScheduler::Stop ( )

Stops scheduling of all tasks.

Definition at line 819 of file task.cc.

◆ task_monitor()

const TaskMonitor* TaskScheduler::task_monitor ( ) const
inline

Definition at line 450 of file task.h.

◆ tbb_arena()

oneapi::tbb::task_arena& TaskScheduler::tbb_arena ( )
inline

returns current TBB arena.

Definition at line 459 of file task.h.

◆ tbb_awake_task()

const TaskTbbKeepAwake* TaskScheduler::tbb_awake_task ( ) const
inline

Definition at line 451 of file task.h.

◆ Terminate()

void TaskScheduler::Terminate ( )

Definition at line 950 of file task.cc.

Here is the call graph for this function:

◆ track_run_time()

bool TaskScheduler::track_run_time ( ) const
inline

Definition at line 425 of file task.h.

◆ use_spawn()

bool TaskScheduler::use_spawn ( ) const
inline

Definition at line 452 of file task.h.

Friends And Related Function Documentation

◆ ConcurrencyScope

friend class ConcurrencyScope
friend

Definition at line 464 of file task.h.

Member Data Documentation

◆ cancel_count_

uint64_t TaskScheduler::cancel_count_
private

Definition at line 511 of file task.h.

◆ done_count_

uint64_t TaskScheduler::done_count_
private

Definition at line 510 of file task.h.

◆ enqueue_count_

uint64_t TaskScheduler::enqueue_count_
private

Definition at line 509 of file task.h.

◆ evm_

EventManager* TaskScheduler::evm_
private

Definition at line 512 of file task.h.

◆ execute_delay_

uint32_t TaskScheduler::execute_delay_
private

Log if time taken to execute exceeds the delay.

Definition at line 507 of file task.h.

◆ hw_thread_count_

int TaskScheduler::hw_thread_count_
private

Definition at line 498 of file task.h.

◆ id_map_

TaskIdMap TaskScheduler::id_map_
private

Definition at line 494 of file task.h.

◆ id_map_mutex_

std::shared_mutex TaskScheduler::id_map_mutex_
private

Definition at line 493 of file task.h.

◆ id_max_

int TaskScheduler::id_max_
private

Definition at line 495 of file task.h.

◆ kVectorGrowSize

const int TaskScheduler::kVectorGrowSize = 16
staticprivate

Definition at line 468 of file task.h.

◆ log_fn_

LogFn TaskScheduler::log_fn_
private

Definition at line 497 of file task.h.

◆ measure_delay_

bool TaskScheduler::measure_delay_
private

Definition at line 501 of file task.h.

◆ mutex_

std::mutex TaskScheduler::mutex_
mutableprivate

Definition at line 488 of file task.h.

◆ running_

bool TaskScheduler::running_
private

Definition at line 489 of file task.h.

◆ schedule_delay_

uint32_t TaskScheduler::schedule_delay_
private

Log if time between enqueue and task-execute exceeds the delay.

Definition at line 504 of file task.h.

◆ seqno_

uint64_t TaskScheduler::seqno_
private

Definition at line 490 of file task.h.

◆ singleton_

boost::scoped_ptr< TaskScheduler > TaskScheduler::singleton_
staticprivate

Definition at line 469 of file task.h.

◆ stop_entry_

TaskEntry* TaskScheduler::stop_entry_
private

Definition at line 484 of file task.h.

◆ task_group_db_

TaskGroupDb TaskScheduler::task_group_db_
private

Definition at line 491 of file task.h.

◆ task_monitor_

TaskMonitor* TaskScheduler::task_monitor_
private

Definition at line 519 of file task.h.

◆ task_scheduler_

oneapi::tbb::task_arena TaskScheduler::task_scheduler_
private

Definition at line 487 of file task.h.

◆ tbb_awake_task_

TaskTbbKeepAwake* TaskScheduler::tbb_awake_task_
private

Definition at line 518 of file task.h.

◆ tbb_global_control_

oneapi::tbb::global_control TaskScheduler::tbb_global_control_
private

Definition at line 486 of file task.h.

◆ ThreadAmpFactor_

int TaskScheduler::ThreadAmpFactor_ = 1
staticprivate

following variable allows one to increase max num of threads used by TBB

Definition at line 516 of file task.h.

◆ track_run_time_

bool TaskScheduler::track_run_time_
private

Definition at line 500 of file task.h.

◆ use_spawn_

bool TaskScheduler::use_spawn_
private

Use spawn() to run a tbb::task instead of enqueue()

Definition at line 483 of file task.h.


The documentation for this class was generated from the following files: