OpenSDN source code
TaskScheduler Class Reference

The TaskScheduler keeps track of what tasks are currently schedulable. When a task is enqueued it is added to the run queue or the pending queue depending as to whether there is a runable or pending task ahead of it that violates the mutual exclusion policies. When tasks exit the scheduler re-examines the tasks on the pending queue which may now be runnable. It is important that this process is efficient such that exit events do not scan tasks that are not waiting on a particular task id or task instance to have a 0 count. More...

#include <task.h>

Collaboration diagram for TaskScheduler:

Public Types

enum  CancelReturnCode { CANCELLED , FAILED , QUEUED }
 
typedef boost::function< void(const char *file_name, uint32_t line_no, const Task *task, const char *description, uint64_t delay)> LogFn
 

Public Member Functions

 TaskScheduler (int thread_count=0)
 TaskScheduler constructor. TBB assumes it can use the "thread" invoking tbb::scheduler can be used for task scheduling. But, in our case we dont want "main" thread to be part of tbb. So, initialize TBB with one thread more than its default. More...
 
 ~TaskScheduler ()
 Frees up the task_entry_db_ allocated for scheduler. More...
 
void Enqueue (Task *task)
 Enqueues a task for running. Starts task if all policy rules are met else puts task in waitq. Enqueueing may may result in the task being immedietly added to the run queue or to a pending queue. Tasks may not be added to the run queue in violation of their exclusion policy. More...
 
void EnqueueUnLocked (Task *task)
 
CancelReturnCode Cancel (Task *task)
 Cancels a Task that can be in RUN/WAIT state. The caller needs to ensure that the task exists when Cancel() is invoked. More...
 
void SetPolicy (int task_id, TaskPolicy &policy)
 Sets the task exclusion policy. Adds policy entries for the task Examples: More...
 
bool GetRunStatus ()
 
int GetTaskId (const std::string &name)
 
std::string GetTaskName (int task_id) const
 
TaskStatsGetTaskGroupStats (int task_id)
 
TaskStatsGetTaskStats (int task_id)
 
TaskStatsGetTaskStats (int task_id, int instance_id)
 
void ClearTaskGroupStats (int task_id)
 
void ClearTaskStats (int task_id)
 
void ClearTaskStats (int task_id, int instance_id)
 
TaskGroupGetTaskGroup (int task_id)
 Get TaskGroup for a task_id. Grows task_entry_db_ if necessary. More...
 
TaskGroupQueryTaskGroup (int task_id)
 Query TaskGroup for a task_id.Assumes valid entry is present for task_id. More...
 
bool IsTaskGroupEmpty (int task_id) const
 Check if there are any Tasks in the given TaskGroup. Assumes that all task ids are mutually exclusive with bgp::Config. More...
 
TaskEntryGetTaskEntry (int task_id, int instance_id)
 Get TaskGroup for a task_id. Grows task_entry_db_ if necessary. More...
 
TaskEntryQueryTaskEntry (int task_id, int instance_id)
 Query TaskEntry for a task-id and task-instance. More...
 
void OnTaskExit (Task *task)
 Method invoked on exit of a Task. Exit of a task can potentially start tasks in pendingq. More...
 
void Stop ()
 Stops scheduling of all tasks. More...
 
void Start ()
 Starts scheduling of all tasks. More...
 
void Print ()
 Debug print routine. More...
 
bool IsEmpty (bool running_only=false)
 Returns true if there are no tasks running and/or enqueued If running_only is true, enqueued tasks are ignored i.e. return true if there are no running tasks. Ignore TaskGroup or TaskEntry if it is disabled. More...
 
void Terminate ()
 
int HardwareThreadCount ()
 
uint64_t enqueue_count () const
 
uint64_t done_count () const
 
uint64_t cancel_count () const
 
void SetMaxThreadCount (int n)
 Force number of threads. More...
 
void GetSandeshData (SandeshTaskScheduler *resp, bool summary)
 
void Log (const char *file_name, uint32_t line_no, const Task *task, const char *description, uint64_t delay)
 
void RegisterLog (LogFn fn)
 
void SetTrackRunTime (bool value)
 
bool track_run_time () const
 
void EnableLatencyThresholds (uint32_t execute, uint32_t schedule)
 Enable logging of tasks exceeding configured latency. More...
 
uint32_t schedule_delay () const
 
uint32_t execute_delay () const
 
bool measure_delay () const
 
void SetLatencyThreshold (const std::string &name, uint32_t execute, uint32_t schedule)
 
uint32_t schedule_delay (Task *task) const
 
uint32_t execute_delay (Task *task) const
 
void set_event_manager (EventManager *evm)
 
void DisableTaskGroup (int task_id)
 
void EnableTaskGroup (int task_id)
 
void DisableTaskEntry (int task_id, int instance_id)
 
void EnableTaskEntry (int task_id, int instance_id)
 
void ModifyTbbKeepAwakeTimeout (uint32_t timeout)
 
void EnableMonitor (EventManager *evm, uint64_t tbb_keepawake_time_msec, uint64_t inactivity_time_msec, uint64_t poll_interval_msec)
 Enable Task monitoring. More...
 
const TaskMonitortask_monitor () const
 
const TaskTbbKeepAwaketbb_awake_task () const
 
bool use_spawn () const
 

Static Public Member Functions

static void Initialize (uint32_t thread_count=0, EventManager *evm=NULL)
 
static TaskSchedulerGetInstance ()
 
static int GetThreadCount (int thread_count=0)
 Get number of tbb worker threads. For testing purposes only. Limit the number of tbb worker threads. More...
 
static bool ShouldUseSpawn ()
 
static int GetDefaultThreadCount ()
 
static void SetThreadAmpFactor (int n)
 following function allows one to increase max num of threads used by TBB More...
 

Private Types

typedef std::vector< TaskGroup * > TaskGroupDb
 
typedef std::map< std::string, int > TaskIdMap
 

Private Member Functions

void SetRunningTask (Task *)
 This function should not be called in production code. It is only for unit testing to control current running task This function modifies the running task as specified by the input. More...
 
void ClearRunningTask ()
 
void WaitForTerminateCompletion ()
 
int CountThreadsPerPid (pid_t pid)
 Platfrom-dependent subroutine in Linux and FreeBSD implementations, used only in TaskScheduler::WaitForTerminateCompletion(). In Linux, make sure that all the [tbb] threads launched have completely exited. We do so by looking for the Threads count of this process in /proc/<pid>/status. More...
 
 DISALLOW_COPY_AND_ASSIGN (TaskScheduler)
 

Private Attributes

bool use_spawn_
 Use spawn() to run a tbb::task instead of enqueue() More...
 
TaskEntrystop_entry_
 
tbb::task_scheduler_init task_scheduler_
 
tbb::mutex mutex_
 
bool running_
 
uint64_t seqno_
 
TaskGroupDb task_group_db_
 
tbb::reader_writer_lock id_map_mutex_
 
TaskIdMap id_map_
 
int id_max_
 
LogFn log_fn_
 
int hw_thread_count_
 
bool track_run_time_
 
bool measure_delay_
 
uint32_t schedule_delay_
 Log if time between enqueue and task-execute exceeds the delay. More...
 
uint32_t execute_delay_
 Log if time taken to execute exceeds the delay. More...
 
uint64_t enqueue_count_
 
uint64_t done_count_
 
uint64_t cancel_count_
 
EventManagerevm_
 
TaskTbbKeepAwaketbb_awake_task_
 
TaskMonitortask_monitor_
 

Static Private Attributes

static const int kVectorGrowSize = 16
 
static boost::scoped_ptr< TaskSchedulersingleton_
 
static int ThreadAmpFactor_ = 1
 following variable allows one to increase max num of threads used by TBB More...
 

Friends

class ConcurrencyScope
 

Detailed Description

The TaskScheduler keeps track of what tasks are currently schedulable. When a task is enqueued it is added to the run queue or the pending queue depending as to whether there is a runable or pending task ahead of it that violates the mutual exclusion policies. When tasks exit the scheduler re-examines the tasks on the pending queue which may now be runnable. It is important that this process is efficient such that exit events do not scan tasks that are not waiting on a particular task id or task instance to have a 0 count.

Definition at line 302 of file task.h.

Member Typedef Documentation

◆ LogFn

typedef boost::function<void(const char *file_name, uint32_t line_no, const Task *task, const char *description, uint64_t delay)> TaskScheduler::LogFn

Definition at line 306 of file task.h.

◆ TaskGroupDb

typedef std::vector<TaskGroup *> TaskScheduler::TaskGroupDb
private

Definition at line 456 of file task.h.

◆ TaskIdMap

typedef std::map<std::string, int> TaskScheduler::TaskIdMap
private

Definition at line 457 of file task.h.

Member Enumeration Documentation

◆ CancelReturnCode

Enumerator
CANCELLED 
FAILED 
QUEUED 

Definition at line 329 of file task.h.

Constructor & Destructor Documentation

◆ TaskScheduler()

TaskScheduler::TaskScheduler ( int  thread_count = 0)

TaskScheduler constructor. TBB assumes it can use the "thread" invoking tbb::scheduler can be used for task scheduling. But, in our case we dont want "main" thread to be part of tbb. So, initialize TBB with one thread more than its default.

Definition at line 451 of file task.cc.

Here is the call graph for this function:

◆ ~TaskScheduler()

TaskScheduler::~TaskScheduler ( )

Frees up the task_entry_db_ allocated for scheduler.

Definition at line 462 of file task.cc.

Member Function Documentation

◆ Cancel()

TaskScheduler::CancelReturnCode TaskScheduler::Cancel ( Task task)

Cancels a Task that can be in RUN/WAIT state. The caller needs to ensure that the task exists when Cancel() is invoked.

Definition at line 705 of file task.cc.

Here is the call graph for this function:

◆ cancel_count()

uint64_t TaskScheduler::cancel_count ( ) const
inline

Definition at line 411 of file task.h.

◆ ClearRunningTask()

void TaskScheduler::ClearRunningTask ( )
private

Definition at line 1012 of file task.cc.

◆ ClearTaskGroupStats()

void TaskScheduler::ClearTaskGroupStats ( int  task_id)

Definition at line 881 of file task.cc.

Here is the call graph for this function:

◆ ClearTaskStats() [1/2]

void TaskScheduler::ClearTaskStats ( int  task_id)

Definition at line 889 of file task.cc.

Here is the call graph for this function:

◆ ClearTaskStats() [2/2]

void TaskScheduler::ClearTaskStats ( int  task_id,
int  instance_id 
)

Definition at line 897 of file task.cc.

Here is the call graph for this function:

◆ CountThreadsPerPid()

int TaskScheduler::CountThreadsPerPid ( pid_t  pid)
private

Platfrom-dependent subroutine in Linux and FreeBSD implementations, used only in TaskScheduler::WaitForTerminateCompletion(). In Linux, make sure that all the [tbb] threads launched have completely exited. We do so by looking for the Threads count of this process in /proc/<pid>/status.

In FreeBSD use libprocstat to check how many threads is running in specific process.

Definition at line 929 of file task.cc.

◆ DisableTaskEntry()

void TaskScheduler::DisableTaskEntry ( int  task_id,
int  instance_id 
)

Definition at line 1038 of file task.cc.

Here is the call graph for this function:

◆ DisableTaskGroup()

void TaskScheduler::DisableTaskGroup ( int  task_id)

Definition at line 1021 of file task.cc.

Here is the call graph for this function:

◆ DISALLOW_COPY_AND_ASSIGN()

TaskScheduler::DISALLOW_COPY_AND_ASSIGN ( TaskScheduler  )
private

◆ done_count()

uint64_t TaskScheduler::done_count ( ) const
inline

Definition at line 410 of file task.h.

◆ EnableLatencyThresholds()

void TaskScheduler::EnableLatencyThresholds ( uint32_t  execute,
uint32_t  schedule 
)

Enable logging of tasks exceeding configured latency.

Definition at line 602 of file task.cc.

◆ EnableMonitor()

void TaskScheduler::EnableMonitor ( EventManager evm,
uint64_t  tbb_keepawake_time_msec,
uint64_t  inactivity_time_msec,
uint64_t  poll_interval_msec 
)

Enable Task monitoring.

Definition at line 518 of file task.cc.

Here is the call graph for this function:

◆ EnableTaskEntry()

void TaskScheduler::EnableTaskEntry ( int  task_id,
int  instance_id 
)

Definition at line 1043 of file task.cc.

Here is the call graph for this function:

◆ EnableTaskGroup()

void TaskScheduler::EnableTaskGroup ( int  task_id)

Definition at line 1031 of file task.cc.

Here is the call graph for this function:

◆ Enqueue()

void TaskScheduler::Enqueue ( Task task)

Enqueues a task for running. Starts task if all policy rules are met else puts task in waitq. Enqueueing may may result in the task being immedietly added to the run queue or to a pending queue. Tasks may not be added to the run queue in violation of their exclusion policy.

Definition at line 642 of file task.cc.

Here is the call graph for this function:

◆ enqueue_count()

uint64_t TaskScheduler::enqueue_count ( ) const
inline

Definition at line 409 of file task.h.

◆ EnqueueUnLocked()

void TaskScheduler::EnqueueUnLocked ( Task task)

Definition at line 648 of file task.cc.

Here is the call graph for this function:

◆ execute_delay() [1/2]

uint32_t TaskScheduler::execute_delay ( ) const
inline

Definition at line 426 of file task.h.

◆ execute_delay() [2/2]

uint32_t TaskScheduler::execute_delay ( Task task) const

Definition at line 548 of file task.cc.

◆ GetDefaultThreadCount()

int TaskScheduler::GetDefaultThreadCount ( )
static

Definition at line 436 of file task.cc.

◆ GetInstance()

TaskScheduler * TaskScheduler::GetInstance ( )
static

Definition at line 554 of file task.cc.

Here is the call graph for this function:

◆ GetRunStatus()

bool TaskScheduler::GetRunStatus ( )
inline

Definition at line 351 of file task.h.

◆ GetSandeshData()

void TaskScheduler::GetSandeshData ( SandeshTaskScheduler *  resp,
bool  summary 
)

Definition at line 1628 of file task.cc.

Here is the call graph for this function:

◆ GetTaskEntry()

TaskEntry * TaskScheduler::GetTaskEntry ( int  task_id,
int  instance_id 
)

Get TaskGroup for a task_id. Grows task_entry_db_ if necessary.

Definition at line 590 of file task.cc.

Here is the call graph for this function:

◆ GetTaskGroup()

TaskGroup * TaskScheduler::GetTaskGroup ( int  task_id)

Get TaskGroup for a task_id. Grows task_entry_db_ if necessary.

Definition at line 561 of file task.cc.

◆ GetTaskGroupStats()

TaskStats * TaskScheduler::GetTaskGroupStats ( int  task_id)

Definition at line 905 of file task.cc.

Here is the call graph for this function:

◆ GetTaskId()

int TaskScheduler::GetTaskId ( const std::string &  name)

Definition at line 862 of file task.cc.

◆ GetTaskName()

std::string TaskScheduler::GetTaskName ( int  task_id) const

Definition at line 852 of file task.cc.

◆ GetTaskStats() [1/2]

TaskStats * TaskScheduler::GetTaskStats ( int  task_id)

Definition at line 913 of file task.cc.

Here is the call graph for this function:

◆ GetTaskStats() [2/2]

TaskStats * TaskScheduler::GetTaskStats ( int  task_id,
int  instance_id 
)

Definition at line 921 of file task.cc.

Here is the call graph for this function:

◆ GetThreadCount()

int TaskScheduler::GetThreadCount ( int  thread_count = 0)
static

Get number of tbb worker threads. For testing purposes only. Limit the number of tbb worker threads.

Definition at line 414 of file task.cc.

◆ HardwareThreadCount()

int TaskScheduler::HardwareThreadCount ( )
inline

Definition at line 400 of file task.h.

◆ Initialize()

void TaskScheduler::Initialize ( uint32_t  thread_count = 0,
EventManager evm = NULL 
)
static

Definition at line 485 of file task.cc.

Here is the call graph for this function:

◆ IsEmpty()

bool TaskScheduler::IsEmpty ( bool  running_only = false)

Returns true if there are no tasks running and/or enqueued If running_only is true, enqueued tasks are ignored i.e. return true if there are no running tasks. Ignore TaskGroup or TaskEntry if it is disabled.

Definition at line 829 of file task.cc.

Here is the call graph for this function:

◆ IsTaskGroupEmpty()

bool TaskScheduler::IsTaskGroupEmpty ( int  task_id) const

Check if there are any Tasks in the given TaskGroup. Assumes that all task ids are mutually exclusive with bgp::Config.

Definition at line 581 of file task.cc.

Here is the call graph for this function:

◆ Log()

void TaskScheduler::Log ( const char *  file_name,
uint32_t  line_no,
const Task task,
const char *  description,
uint64_t  delay 
)

Definition at line 530 of file task.cc.

◆ measure_delay()

bool TaskScheduler::measure_delay ( ) const
inline

Definition at line 428 of file task.h.

◆ ModifyTbbKeepAwakeTimeout()

void TaskScheduler::ModifyTbbKeepAwakeTimeout ( uint32_t  timeout)

Definition at line 512 of file task.cc.

Here is the call graph for this function:

◆ OnTaskExit()

void TaskScheduler::OnTaskExit ( Task task)

Method invoked on exit of a Task. Exit of a task can potentially start tasks in pendingq.

Definition at line 768 of file task.cc.

Here is the call graph for this function:

◆ Print()

void TaskScheduler::Print ( )

Debug print routine.

Definition at line 814 of file task.cc.

Here is the call graph for this function:

◆ QueryTaskEntry()

TaskEntry * TaskScheduler::QueryTaskEntry ( int  task_id,
int  instance_id 
)

Query TaskEntry for a task-id and task-instance.

Definition at line 595 of file task.cc.

Here is the call graph for this function:

◆ QueryTaskGroup()

TaskGroup * TaskScheduler::QueryTaskGroup ( int  task_id)

Query TaskGroup for a task_id.Assumes valid entry is present for task_id.

Definition at line 577 of file task.cc.

◆ RegisterLog()

void TaskScheduler::RegisterLog ( LogFn  fn)

Definition at line 538 of file task.cc.

◆ schedule_delay() [1/2]

uint32_t TaskScheduler::schedule_delay ( ) const
inline

Definition at line 425 of file task.h.

◆ schedule_delay() [2/2]

uint32_t TaskScheduler::schedule_delay ( Task task) const

Definition at line 542 of file task.cc.

◆ set_event_manager()

void TaskScheduler::set_event_manager ( EventManager evm)

Definition at line 500 of file task.cc.

Here is the call graph for this function:

◆ SetLatencyThreshold()

void TaskScheduler::SetLatencyThreshold ( const std::string &  name,
uint32_t  execute,
uint32_t  schedule 
)

Definition at line 609 of file task.cc.

Here is the call graph for this function:

◆ SetMaxThreadCount()

void TaskScheduler::SetMaxThreadCount ( int  n)

Force number of threads.

◆ SetPolicy()

void TaskScheduler::SetPolicy ( int  task_id,
TaskPolicy policy 
)

Sets the task exclusion policy. Adds policy entries for the task Examples:

  • Policy <tid0> => <tid1, -1> <tid2, inst2> will result in following:
    • task_db_[tid0] : Rule <tid1, -1> is added to policyq
    • task_group_db_[tid0, inst2] : Rule <tid2, inst2> is added to policyq
  • The symmetry of policy will result in following additional rules,
    • task_db_[tid1] : Rule <tid0, -1> is added to policyq
    • task_group_db_[tid2, inst2] : Rule <tid0, inst2> is added to policyq

Definition at line 617 of file task.cc.

Here is the call graph for this function:

◆ SetRunningTask()

void TaskScheduler::SetRunningTask ( Task unit_test)
private

This function should not be called in production code. It is only for unit testing to control current running task This function modifies the running task as specified by the input.

Definition at line 1007 of file task.cc.

Here is the call graph for this function:

◆ SetThreadAmpFactor()

void TaskScheduler::SetThreadAmpFactor ( int  n)
static

following function allows one to increase max num of threads used by TBB

Definition at line 1017 of file task.cc.

◆ SetTrackRunTime()

void TaskScheduler::SetTrackRunTime ( bool  value)
inline

Definition at line 420 of file task.h.

◆ ShouldUseSpawn()

bool TaskScheduler::ShouldUseSpawn ( )
static

Definition at line 440 of file task.cc.

◆ Start()

void TaskScheduler::Start ( )

Starts scheduling of all tasks.

Definition at line 804 of file task.cc.

Here is the call graph for this function:

◆ Stop()

void TaskScheduler::Stop ( )

Stops scheduling of all tasks.

Definition at line 798 of file task.cc.

◆ task_monitor()

const TaskMonitor* TaskScheduler::task_monitor ( ) const
inline

Definition at line 446 of file task.h.

◆ tbb_awake_task()

const TaskTbbKeepAwake* TaskScheduler::tbb_awake_task ( ) const
inline

Definition at line 447 of file task.h.

◆ Terminate()

void TaskScheduler::Terminate ( )

Definition at line 984 of file task.cc.

Here is the call graph for this function:

◆ track_run_time()

bool TaskScheduler::track_run_time ( ) const
inline

Definition at line 421 of file task.h.

◆ use_spawn()

bool TaskScheduler::use_spawn ( ) const
inline

Definition at line 448 of file task.h.

◆ WaitForTerminateCompletion()

void TaskScheduler::WaitForTerminateCompletion ( )
private

Definition at line 958 of file task.cc.

Here is the call graph for this function:

Friends And Related Function Documentation

◆ ConcurrencyScope

friend class ConcurrencyScope
friend

Definition at line 455 of file task.h.

Member Data Documentation

◆ cancel_count_

uint64_t TaskScheduler::cancel_count_
private

Definition at line 513 of file task.h.

◆ done_count_

uint64_t TaskScheduler::done_count_
private

Definition at line 512 of file task.h.

◆ enqueue_count_

uint64_t TaskScheduler::enqueue_count_
private

Definition at line 511 of file task.h.

◆ evm_

EventManager* TaskScheduler::evm_
private

Definition at line 514 of file task.h.

◆ execute_delay_

uint32_t TaskScheduler::execute_delay_
private

Log if time taken to execute exceeds the delay.

Definition at line 509 of file task.h.

◆ hw_thread_count_

int TaskScheduler::hw_thread_count_
private

Definition at line 500 of file task.h.

◆ id_map_

TaskIdMap TaskScheduler::id_map_
private

Definition at line 496 of file task.h.

◆ id_map_mutex_

tbb::reader_writer_lock TaskScheduler::id_map_mutex_
private

Definition at line 495 of file task.h.

◆ id_max_

int TaskScheduler::id_max_
private

Definition at line 497 of file task.h.

◆ kVectorGrowSize

const int TaskScheduler::kVectorGrowSize = 16
staticprivate

Definition at line 459 of file task.h.

◆ log_fn_

LogFn TaskScheduler::log_fn_
private

Definition at line 499 of file task.h.

◆ measure_delay_

bool TaskScheduler::measure_delay_
private

Definition at line 503 of file task.h.

◆ mutex_

tbb::mutex TaskScheduler::mutex_
mutableprivate

Definition at line 490 of file task.h.

◆ running_

bool TaskScheduler::running_
private

Definition at line 491 of file task.h.

◆ schedule_delay_

uint32_t TaskScheduler::schedule_delay_
private

Log if time between enqueue and task-execute exceeds the delay.

Definition at line 506 of file task.h.

◆ seqno_

uint64_t TaskScheduler::seqno_
private

Definition at line 492 of file task.h.

◆ singleton_

boost::scoped_ptr< TaskScheduler > TaskScheduler::singleton_
staticprivate

Definition at line 460 of file task.h.

◆ stop_entry_

TaskEntry* TaskScheduler::stop_entry_
private

Definition at line 487 of file task.h.

◆ task_group_db_

TaskGroupDb TaskScheduler::task_group_db_
private

Definition at line 493 of file task.h.

◆ task_monitor_

TaskMonitor* TaskScheduler::task_monitor_
private

Definition at line 521 of file task.h.

◆ task_scheduler_

tbb::task_scheduler_init TaskScheduler::task_scheduler_
private

Definition at line 489 of file task.h.

◆ tbb_awake_task_

TaskTbbKeepAwake* TaskScheduler::tbb_awake_task_
private

Definition at line 520 of file task.h.

◆ ThreadAmpFactor_

int TaskScheduler::ThreadAmpFactor_ = 1
staticprivate

following variable allows one to increase max num of threads used by TBB

Definition at line 518 of file task.h.

◆ track_run_time_

bool TaskScheduler::track_run_time_
private

Definition at line 502 of file task.h.

◆ use_spawn_

bool TaskScheduler::use_spawn_
private

Use spawn() to run a tbb::task instead of enqueue()

Definition at line 486 of file task.h.


The documentation for this class was generated from the following files: