Up until the latest update to FreeBSD 12.1-RELEASE-p7, limiting jails to a few cores was two lines in the jail.conf config file. After the update, I had a jail refuse to start and display the following error:
cpuset: setaffinity: Resource deadlock avoided

In my jail.conf I use the exec.poststart entry to limit the CPUs to which each jail can spawn:

...
exec.poststart    = "cpuset -c -l ${cpuset} -j ${name}";
...

Farther down in the same file, I define the jail's allowed CPUs using a variable:

...
$cpuset = "7-8";
...

In this example, all processes on that specific jail are spawned on cpu 7 or 8. Easy.


The error above only appeared on a single jail, the one hosting my internal bind9 DNS services. After a few Google searches, it appears I'm not alone. Same service, same error.

The poster goes into an explanation as to how the bind daemon, named, will spawn one process for each core on the system. They go on to publish a script, called by the same exec.poststart stage, that appears to capture all processes spawned by that jail. Then it continues to limit the jail's processes to the desired cores. Or at least I think that's what it says. It's in Russian.


The Fix

My takeaway from the research above was the part about named spawning a process for each core. As a test, I first let the jail run on all cores by removing the cpuset restraints. This allowed the jail to start, but as expected, named processes were started and bound to each cpu core.

Next, I tried disabling the named service, restoring the cpuset constraints, then I tried to start the jail. Success! The jail would start. That confirms named is the culprit.

Next, I logged into the jail and started named manually: Success! It started and is only bound to the desired cores!

And now for the super elaborate fix: crontab -e Yup, I know it's hack-y but it does the job. I left the service disabled in /etc/rc.cond.local and instead started it via cron:

@reboot sleep 5 && service named onestart

This will allow the jail to start, let the exec.poststart phase execute, then the process starts after constraints are put in place.