I am leveraging the new Sched-ext capability in the kernel to write an ebpf scheduler using C. I have two sets of CPUs: the fast group and the slow group. When a task first arrives, it’s scheduled on one of the fast CPUs (using the select_cpu callback.) And it’s given a 100ms slice (using scx_bpf_dispatch) to run on this CPU.
If this slice is exhausted the first time, I want this task to be taken off this CPU and rescheduled to another cpu in the slow group. What helper function in ebpf and scx can I use to do so? I can’t find anything that could take a task off a local dsq once it has been inserted there.
As far as I found out, the task can’t be taken off the DSQ in the task_tick function. What can be done is mark the task’s pid (through a bpf map or other global data structure) and dispatch it to another DSQ using scx_bpf_dispatch when the enqueue callback is called. When the slice is exhausted, if SCX_OPS_ENQ_LAST is enabled as an ops flag OR the task is not the only one on this CPU, the task is preempted and .enqueue is called on it. It is there that it can be moved to another DSQ or BPF structure.