{"revision": {"id": "f3de6cd1-2f95-11f1-b7c7-e86a64d24d78", "node_id": "f3dd655c-2f95-11f1-ab31-e86a64d24d78", "user_id": "edc3f576-2f95-11f1-900f-e86a64d24d78", "author": "foxhop", "data": "Salt Stack Piggy Back Encrypted ZMQ Event Bus\r\n##################################################\r\n\r\nSome code examples that piggy back on Salt Stack's encrypted ZMQ event bus.\r\nThese code examples show how 3rd party scripts on each side may communicate over \r\nsecure and fast transport!\r\n\r\nRequires:\r\n\r\n* the salt-minion alive and running on the minion host\r\n* the salt-master alive and running on the master host \r\n\r\n.. contents::\r\n\r\n\r\nFire event to minion bus and listen to minion bus\r\n====================================================\r\nIn this example we write a listener/subscriber script that connects to the minion bus.\r\nWe then write a emitter/publisher script that connects to the minion bus and posts events.\r\n\r\nlisten_to_minion_bus.py\r\n------------------------------\r\n3rd Party Subscriber / Listener:\r\n\r\n.. code-block:: python\r\n\r\n # needed for config to opts processing\r\n import os\r\n import salt.syspaths as syspaths\r\n import salt.config\r\n\r\n # get opts from minion config file, this function also looks in drop dir!\r\n opts = salt.config.minion_config(os.path.join(syspaths.CONFIG_DIR, 'minion'))\r\n\r\n # debug to STDOUT\r\n import salt.log\r\n salt.log.setup_console_logger('all')\r\n\r\n # event libary for events over ZMQ\r\n import salt.utils.event\r\n\r\n # opts must have the valid minion ID else it binds to invalid socket\r\n event = salt.utils.event.MinionEvent(**opts)\r\n\r\n tag = 'mytag'\r\n\r\n print('Listening for events tagged \\'{}\\' on Salt Minion bus.'.format(tag))\r\n\r\n # generator iterator yields events forever, we filter on tag\r\n for data in event.iter_events(tag=tag):\r\n     print(data)\r\n\r\nemit_to_minion_bus.py\r\n-------------------------------\r\n3rd Party Publisher / Emitter:\r\n\r\n.. code-block:: python\r\n\r\n # needed for config to opts processing\r\n import os\r\n import salt.syspaths as syspaths\r\n import salt.config\r\n\r\n # get opts from minion config file, this function also looks in drop dir!\r\n opts = salt.config.minion_config(os.path.join(syspaths.CONFIG_DIR, 'minion'))\r\n\r\n # debug to STDOUT\r\n import salt.log\r\n salt.log.setup_console_logger('all')\r\n\r\n # event libary for events over ZMQ\r\n import salt.utils.event\r\n\r\n tag = 'mytag'\r\n data = {'message':'a drop in the bucket'}\r\n\r\n payload = {\r\n   'id': opts['id'],\r\n   'tag': tag,\r\n   'data': data,\r\n }\r\n \r\n print (payload)\r\n\r\n # opts must valid minion ID else it binds to invalid socket\r\n event = salt.utils.event.SaltEvent('minion', **opts)\r\n\r\n # Fire event payload with tag\r\n event.fire_event(payload, tag)\r\n\r\nNotes on the implementation\r\n----------------------------------\r\nNotice how I use `event.MinionEvent` in listener and `event.SaltEvent` in emitter? \r\nThe `event.MinionEvent` is just a very light weight subclass of `event.SaltEvent` so you can use\r\neither or...\r\n\r\nThe configuration code could be reduced to static variables, for instance replace -\r\n\r\n.. code-block:: python\r\n\r\n # needed for config to opts processing\r\n import os\r\n import salt.syspaths as syspaths\r\n import salt.config\r\n\r\n # get opts from minion config file, this function also looks in drop dir!\r\n opts = salt.config.minion_config(os.path.join(syspaths.CONFIG_DIR, 'minion'))\r\n\r\nwith this \r\n\r\n.. code-block:: python\r\n\r\n opts = {'sock_dir':'/var/run/salt/minion', 'id':'id-of-this-minion'}\r\n\r\nFire event to master bus and listen to master bus\r\n====================================================\r\nIn this example we write a listener/subscriber script that connects to the master bus.\r\nWe then write a emitter/publisher script that connects to the master bus and posts events.\r\n\r\nlisten_to_master_bus.py\r\n--------------------------------\r\n\r\n.. code-block:: python\r\n\r\n # debug logging to STDOUT\r\n import salt.log\r\n salt.log.setup_console_logger('all')\r\n\r\n # event libary for events over ZMQ\r\n import salt.utils.event\r\n\r\n # create event object, attach to master socket ...\r\n event = salt.utils.event.MasterEvent('/var/run/salt/master')\r\n\r\n tag = 'mytag'\r\n\r\n print('Listening for events tagged \\'{}\\' on Salt Master bus.'.format(tag))\r\n\r\n # generator iterator yields events forever, we filter on tag\r\n for data in event.iter_events(tag=tag):\r\n     print(data)\r\n\r\n\r\nemit_to_master_bus.py\r\n---------------------------------\r\n\r\n.. code-block:: python\r\n\r\n # debug logging to STDOUT\r\n import salt.log\r\n salt.log.setup_console_logger('all')\r\n\r\n # event libary for events over ZMQ\r\n import salt.utils.event\r\n\r\n payload = {'sample-msg': 'this is a test',\r\n            'example': 'this is the same test'}\r\n\r\n sock_dir = '/var/run/salt/master'\r\n\r\n # create event object, attach to master socket ...\r\n event = salt.utils.event.SaltEvent('master', sock_dir)\r\n\r\n # post the event\r\n event.fire_event(payload, 'mytag')\r\n\r\n\r\nFire events between minion and master\r\n====================================================\r\n\r\nThere is where I get stuck ...  I want to allow minions to fire events which end up on the master bus ...\r\n\r\nI'm able to do this with `salt-call`, for example:\r\n\r\n.. code-block:: python\r\n\r\n salt-call event.fire_master '{\"data\": \"message for the master\"}' 'mytag'\r\n\r\nFrom My understanding the minion to master communication could work like this:\r\n\r\n.. code-block:: text\r\n\r\n A 3rd party script publishes to minion bus ->\r\n salt-minion daemon sees the properly formatted \"package\" event and runs _fire_master ->\r\n the \"package\" event  ends up on master bus -> \r\n A 3rd party script subscribes to the tag and gains access to the data.\r\n\r\nThe code in `emit-to-minion-bus.py <http://www.foxhop.net/salt-stack-piggy-back-encrypted-zmq-event-bus#emit-to-minion-bus-py>`_ does the first step, but it seems my \"package\" format is incorrect so it never gets forwarded to the master ...\r\n\r\n\r\nideas\r\n============\r\n\r\nI think it could be neat to build a returner which accepted a ZMQ \"channel\" tag.  Then after a remote execution the and ret data could be passed from minion to master on a special channel.  Then we could have a 3rd party script on the salt master that subscribes to this special channel and performs further processing or presents the data in a neat way.\r\n\r\nfor example:\r\n\r\n.. code-block:: text\r\n\r\n salt '*' --return=master_bus --tag=monitoring cmd.run_all '/usr/lib/nagios/plugins/check_procs -w 150 -c 200'\r\n\r\nThe minion would see the remote execution command, run the job, get data back, and return to master on a special tagged channel.\r\n\r\nOn the master we could have a 3rd party script subscribed to the special tagged channel and perform further manipulation or persist the output somewhere.\r\n\r\n\r\nreferences\r\n===============\r\n\r\nhttp://docs.saltstack.com/topics/event/index.html\r\n\r\n\r\n\r\n\r\n", "source_format": "rst", "revision_number": 22, "created": 1393771708000}}