Queue-based Worker Manager

[al-rigazzi]

This PR adds the RequestDispatcher to the MLI. The RequestDispatcher batches inference requests together.

The implementation can be improved, especially by adding:

• Abstraction for Memory, so that Dragon's MemoryPool can be wrapped in a SmartSim class and different types of memory can be injected (esp. at unit testing time)
• More parameters around Torch Threads and intra-op threads
• Queue removal mechanism in RequestDispatcher
• Model removal mechanism if OOM error when loading a model
• Tests for RequestDispatcher, BatchQueue, DeviceManager, and so on.

There is no mechanism to address model versions right now.

We can decide what to do now and what to put up a ticket for.

[AlyssaCote]

LGTM! One tiny comment about potentially removing a timing line, but not worth holding up the approval!

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 623 lines in your changes missing coverage. Please review.

Please upload report for BASE (mli-feature@6d5518b). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
...re/mli/infrastructure/control/requestdispatcher.py	0.00%	254 Missing ⚠️
.../_core/mli/infrastructure/control/workermanager.py	0.00%	95 Missing ⚠️
smartsim/_core/utils/timings.py	0.00%	86 Missing ⚠️
smartsim/_core/mli/infrastructure/worker/worker.py	0.00%	66 Missing ⚠️
...im/_core/mli/infrastructure/worker/torch_worker.py	0.00%	63 Missing ⚠️
.../_core/mli/infrastructure/control/devicemanager.py	0.00%	42 Missing ⚠️
..._core/mli/infrastructure/control/error_handling.py	0.00%	16 Missing ⚠️
smartsim/_core/launcher/dragon/dragonBackend.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff               @@
##             mli-feature     #647   +/-   ##
==============================================
  Coverage               ?   71.34%           
==============================================
  Files                  ?      102           
  Lines                  ?     8525           
  Branches               ?        0           
==============================================
  Hits                   ?     6082           
  Misses                 ?     2443           
  Partials               ?        0           

Files with missing lines	Coverage Δ
smartsim/_core/launcher/dragon/dragonBackend.py	1.96% <0.00%> (ø)
..._core/mli/infrastructure/control/error_handling.py	0.00% <0.00%> (ø)
.../_core/mli/infrastructure/control/devicemanager.py	0.00% <0.00%> (ø)
...im/_core/mli/infrastructure/worker/torch_worker.py	0.00% <0.00%> (ø)
smartsim/_core/mli/infrastructure/worker/worker.py	0.00% <0.00%> (ø)
smartsim/_core/utils/timings.py	0.00% <0.00%> (ø)
.../_core/mli/infrastructure/control/workermanager.py	0.00% <0.00%> (ø)
...re/mli/infrastructure/control/requestdispatcher.py	0.00% <0.00%> (ø)

[mellis13]

Fantastic first implementation of the queue-based architecture. Thanks!