changeset 414:2d7ab98d6266

better lightsc.py and documentation
author Louis Opter <kalessin@kalessin.fr>
date Sat, 02 Jan 2016 22:13:26 +0100
parents 9067c2da9572
children 9d9c937212cb
files fix_lightscpy_readloop.patch
diffstat 1 files changed, 211 insertions(+), 31 deletions(-) [+]
line wrap: on
line diff
--- a/fix_lightscpy_readloop.patch	Fri Jan 01 13:24:12 2016 +0100
+++ b/fix_lightscpy_readloop.patch	Sat Jan 02 22:13:26 2016 +0100
@@ -1,16 +1,174 @@
 # HG changeset patch
 # Parent  c8614ad2dc1133c8d50e974afd66d84231b41c78
-Fix lightsc.py's read loop
+Fix lightsc.py's read loop and document it
 
 It now handles arbitrarily large and partial responses properly.
 
-We do need to do non-blocking IO in case the last received buffer comes
-back full in this case doing another read would block the whole thing.
-
+diff --git a/docs/protocol.rst b/docs/protocol.rst
+--- a/docs/protocol.rst
++++ b/docs/protocol.rst
+@@ -1,7 +1,12 @@
+ The lights daemon protocol
+ ==========================
+ 
+-The lightsd protocol is implemented on top of `JSON-RPC 2.0`_.
++The lightsd protocol is implemented on top of `JSON-RPC 2.0`_. This section
++covers the available methods and how to target bulbs.
++
++Since lightsd implements JSON-RPC without any kind of framing like it usually is
++the case (using HTTP), this section also explains how to implement your own
++lightsd client in `Writing a client for lightsd`_.
+ 
+ .. _JSON-RPC 2.0: http://www.jsonrpc.org/specification
+ 
+@@ -19,7 +24,7 @@
+ | ``#TagName``                | targets bulbs tagged with *TagName*            |
+ +-----------------------------+------------------------------------------------+
+ | ``124f31a5``                | directly target the bulb with the given id     |
+-|                             | (mac addr, see below)                          |
++|                             | (that's the bulb mac address, see below)       |
+ +-----------------------------+------------------------------------------------+
+ | ``label``                   | directly target the bulb with the given Label  |
+ +-----------------------------+------------------------------------------------+
+@@ -52,21 +57,23 @@
+ 
+    Power off the given bulb(s) with an optional transition.
+ 
+-   :param int transition: The time in ms it will take for the bulb to turn off.
++   :param int transition: Optional time in ms it will take for the bulb to turn
++                          off.
+ 
+ .. function:: power_on(target[, transition])
+ 
+    Power on the given bulb(s) with an optional transition.
+ 
+-   :param int transition: The time in ms it will take for the bulb to turn on.
++   :param int transition: Optional time in ms it will take for the bulb to turn
++                          on.
+ 
+ .. function:: power_toggle(target[, transition])
+ 
+    Power on (if they are off) or power off (if they are on) the given bulb(s)
+    with an optional transition.
+ 
+-   :param int transition: The time in ms it will take for the bulb to turn on
+-                          off.
++   :param int transition: Optional time in ms it will take for the bulb to turn
++                          on off.
+ 
+ .. function:: set_light_from_hsbk(target, h, s, b, k, transition)
+ 
+@@ -152,9 +159,105 @@
+ 
+       untag("#myexistingtag", "myexistingtag")
+ 
++Writing a client for lightsd
++----------------------------
++
++lightsd does JSON-RPC directly over TCP, requests and responses aren't framed in
++any way like it is usually done by using HTTP.
++
++This means that you will very likely need to write a JSON-RPC client
++specifically for lightsd. You're actually encouraged to do that as lightsd will
++probably augment JSON-RPC via lightsd specific `JSON-RPC extensions`_ in the
++future.
++
++.. _JSON-RPC extensions: http://www.jsonrpc.org/specification#extensions
++
++JSON-RPC over TCP
++~~~~~~~~~~~~~~~~~
++
++JSON-RPC works in a request/response fashion: the socket (network connection) is
++never used in a full-duplex fashion (data never flows in both direction at the
++same time):
++
++#. Write (send) a request on the socket;
++#. Read (receive) the response on the socket;
++#. Repeat.
++
++Writing the request is easy: do successive write (send) calls until you have
++successfully sent the whole request. The next step (reading/receiving) is a bit
++more complex. And that said, if the response isn't useful to you, you can ask
++lightsd to omit it by turning your request into a `notification`_: if you remove
++the JSON-RPC id, then you can just send your requests (now notifications) on the
++socket in a fire and forget fashion.
++
++.. _notification: http://www.jsonrpc.org/specification#notification
++
++Otherwise to successfully read and decode JSON-RPC over TCP you will need to
++implement your own read loop, the algorithm follows. It focuses on the low-level
++details, adapt it for the language and platform you are using:
++
++#. Prepare an empty buffer that you can grow, we will accumulate received data
++   in it;
++#. Start an infinite loop and start a read (receive) for a chunk of data (e.g:
++   4KiB), accumulate the received data in the previous buffer, then try to
++   interpret the data as JSON:
++
++   - if valid JSON can be decoded then break out of the loop;
++   - else data is missing and continue the loop;
++#. Decode the JSON data.
++
++Here is a complete Python 3 request/response example:
++
++.. code-block:: python
++   :linenos:
++
++   import json
++   import socket
++   import uuid
++
++   READ_SIZE = 4096
++   ENCODING = "utf-8"
++
++   # Connect to lightsd, here using an Unix socket. The rest of the example is
++   # valid for TCP sockets too. Replace /run/lightsd/socket by the output of:
++   # echo $(lightsd --rundir)/socket
++   lightsd_socket = socket.socket(socket.AF_UNIX)
++   lightsd_socket.connect("/run/lightsd/socket")
++   lightsd_socket.settimeout(2)  # seconds
++
++   # Prepare the request:
++   request = json.dumps({
++       "method": "get_light_state",
++       "params": ["*"],
++       "jsonrpc": "2.0",
++       "id": str(uuid.uuid4()),
++   }).encode(ENCODING, "surrogateescape")
++
++   # Send it:
++   lightsd_socket.sendall(request)
++
++   # Prepare an empty buffer to accumulate the received data:
++   response = bytearray()
++   while True:
++       # Read a chunk of data, and accumulate it in the response buffer:
++       response += lightsd_socket.recv(READ_SIZE)
++       try:
++           # Try to load the received the data, we ignore encoding errors
++           # since we only wanna know if the received data is complete.
++           json.loads(response.decode(ENCODING, "ignore"))
++           break  # Decoding was successful, we have received everything.
++       except Exception:
++           continue  # Decoding failed, data must be missing.
++
++   response = response.decode(ENCODING, "surrogateescape")
++   print(json.loads(response))
++
+ Notes
+------
++~~~~~
+ 
++- Use an incremental JSON parser if you have one handy: for responses multiple
++  times the size of your receive window it will let you avoid decoding the whole
++  response at each iteration of the read loop;
+ - lightsd supports batch JSON-RPC requests, use them!
+ 
+ .. vim: set tw=80 spelllang=en spell:
 diff --git a/examples/lightsc.py b/examples/lightsc.py
 --- a/examples/lightsc.py
 +++ b/examples/lightsc.py
-@@ -30,9 +30,11 @@
+@@ -30,6 +30,7 @@
  
  import argparse
  import contextlib
@@ -18,29 +176,26 @@
  import json
  import locale
  import os
-+import select
- import socket
- import subprocess
- import sys
-@@ -42,6 +44,9 @@
+@@ -42,6 +43,10 @@
  
  class LightsClient:
  
 +    READ_SIZE = 4096
++    TIMEOUT = 2  # seconds
 +    ENCODING = "utf-8"
 +
      def __init__(self, url):
          self.url = url
  
-@@ -56,6 +61,7 @@
+@@ -55,6 +60,7 @@
+             self._socket = socket.create_connection((parts.hostname, parts.port))
          else:
              raise ValueError("Unsupported url {}".format(url))
++        self._socket.settimeout(self.TIMEOUT)
  
-+        fcntl.fcntl(self._socket, fcntl.F_SETFL, os.O_NONBLOCK)
          self._pipeline = []
          self._batch = False
- 
-@@ -75,17 +81,27 @@
+@@ -75,16 +81,20 @@
          }
  
      def _execute_payload(self, payload):
@@ -48,30 +203,55 @@
 -        self._socket.send(json.dumps(payload).encode("utf-8"))
 -        # FIXME: proper read loop
 -        response = self._socket.recv(64 * 1024).decode("utf-8")
-+        select.select([], [self._socket], [])
+-        try:
+-            response = json.loads(response)
+-        except Exception:
+-            print("received invalid json: {}".format(response))
 +        payload = json.dumps(payload).encode(self.ENCODING, "surrogateescape")
-+        self._socket.send(payload)
-+
++        self._socket.sendall(payload)
+ 
+-        return response
 +        response = bytearray()
-+        select.select([self._socket], [], [])
 +        while True:
++            response += self._socket.recv(self.READ_SIZE)
 +            try:
-+                part = self._socket.recv(self.READ_SIZE)
-+            except BlockingIOError:
++                json.loads(response.decode(self.ENCODING, "ignore"))
 +                break
-+            if not part:
-+                break
-+            response += part
++            except Exception:
++                continue
 +
 +        response = response.decode(self.ENCODING, "surrogateescape")
-         try:
--            response = json.loads(response)
-+            return json.loads(response)
-         except Exception:
-             print("received invalid json: {}".format(response))
++        return json.loads(response)
  
--        return response
--
      def _jsonrpc_call(self, method, params):
          payload = self._make_payload(method, params)
-         if self._batch:
+@@ -203,11 +213,6 @@
+ 
+ def _drop_to_shell(lightsc):
+     c = lightsc  # noqa
+-    nb = "d073d501a0d5"  # noqa
+-    fugu = "d073d500603b"  # noqa
+-    neko = "d073d5018fb6"  # noqa
+-    middle = "d073d502e530"  # noqa
+-
+     banner = (
+         "Connected to {}, use the variable c to interact with your "
+         "bulbs:\n\n>>> r = c.get_light_state(\"*\")".format(c.url)
+@@ -231,7 +236,7 @@
+         lightsdrundir = subprocess.check_output(["lightsd", "--rundir"])
+     except Exception as ex:
+         print(
+-            "Couldn't infer lightsd's runtime directory is lightsd installed? "
++            "Couldn't infer lightsd's runtime directory, is lightsd installed? "
+             "({})\nTrying build/socket...".format(ex),
+             file=sys.stderr
+         )
+@@ -242,7 +247,7 @@
+         lightsdrundir = lightsdrundir.decode(encoding).strip()
+ 
+     parser = argparse.ArgumentParser(
+-        description="lightsc.py is an interactive lightsd Python client"
++        description="Interactive lightsd Python client"
+     )
+     parser.add_argument(
+         "-u", "--url", type=str,