Cron Improvements, June 20, 2019

[buskamuza]

Agenda:

• Cron improvements proposal - @knowj
• Using queue for cron tasks - @knowj / @kandy / @buskamuza
• Next steps in the project - @dmanners

Please add your topic as a comment to the issue. Use following format:
Topic description and link to PR, if any (duration in min)

[buskamuza]

• @piotrekkaminski : can we reuse existing tools
• @buskamuza : isolation/multi-threading - should we just improve existing "groups" functionality? @knowj : yes. Education for how to setup cron to utilize groups. Revisit existing Magento cron groups to make sure that critical jobs can run in a separate process.
• @kandy : should visualization be implemented in Magento core, while there are 3rd-party extensions available with visualization? @knowj : it becomes more critical to have in core as we move on with Service Isolation
• @knowj : when moving towards using message queue for jobs execution, how transition period will look like? @kandy : there should be no problems, as jobs execution is controlled by Magento framework
• @buskamuza : If moving tasks to message queue, is message queue admin panel enough for visualization? @nuzil : it's not very convenient. @knowj : it should be enough for sysops, but not everybody has them

[buskamuza]

Next steps:

• @knowj : update the proposal. Sync with @buskamuza and prepare for presentation on the next Arch meeting
• @buskamuza : sync with @dmanners on how to organize the community project, where @piotrekkaminski can start writing down the backlog

[maghamed]

hey guys, regarding the visualization mechanism for Cron.

We need to make sure to design it as much decoupled from existing cron implementation as possible.
Data visualization should provide APIs to gather information, and existing cron jobs should rely on these APIs. Thus, if we will decide to make changes in the architecture of cron mechanism later - it would not affect visualization.

Also, we need to consider this mechanism with the same APIs for collecting data / visualization for Async Operations (executed via Message Queue), @nuzil may have more insights here.