Simplified API

Fleshing out this PR with additional polish:

* Initial Sphinx docs
* Test for generator
* A more comprehensive Makefile
* Fix type hints in the generator and in Trio CDP.

The documentation still needs work and I also still need to fix type
hints in the generated code.

Author: mehaase

Makefile

@@ -1,9 +1,29 @@
-.PHONY: test
+.PHONY: docs
 
-publish: test
+default: mypy-generate test-generate generate test-import mypy-cdp test-cdp
+
+docs:
+	$(MAKE) -C docs html
+
+generate:
+	python generator/generate.py
+
+mypy-cdp:
+	mypy trio_cdp/
+
+mypy-generate:
+	mypy generator/
+
+publish: test-import mypy-cdp test-cdp
 	rm -fr dist trio_chrome_devtools_protocol.egg-info
 	python setup.py sdist
 	twine upload dist/*
 
-test:
+test-cdp:
 	pytest tests/ --cov=trio_cdp --cov-report=term-missing
+
+test-generate:
+	pytest generator/
+
+test-import:
+	python -c 'import trio_cdp; print(trio_cdp.accessibility)'

README.md

@@ -4,6 +4,7 @@
 ![Python Versions](https://img.shields.io/pypi/pyversions/trio-chrome-devtools-protocol)
 ![MIT License](https://img.shields.io/github/license/HyperionGray/trio-chrome-devtools-protocol.svg)
 [![Build Status](https://img.shields.io/travis/com/HyperionGray/trio-chrome-devtools-protocol.svg?branch=master)](https://travis-ci.com/HyperionGray/trio-chrome-devtools-protocol)
+[![Read the Docs](https://img.shields.io/readthedocs/trio-cdp.svg)](https://trio-cdp.readthedocs.io)
 
 This Python library performs remote control of any web browser that implements
 the Chrome DevTools Protocol. It is built using the type wrappers in
@@ -12,180 +13,32 @@ I/O using [Trio](https://trio.readthedocs.io/). This library handles the
 WebSocket negotiation and session management, allowing you to transparently
 multiplex commands, responses, and events over a single connection.
 
-The example demonstrates the salient features of the library.
+The example below demonstrates the salient features of the library by navigating to a
+web page and extracting the document title.
 
 ```python
+from trio_cdp import open_cdp, page, dom
+
 async with open_cdp(cdp_url) as conn:
     # Find the first available target (usually a browser tab).
-    targets = await conn.execute(target.get_targets())
+    targets = await target.get_targets()
     target_id = targets[0].id
 
     # Create a new session with the chosen target.
-    session = await conn.open_session(target_id)
-
-    # Navigate to a website.
-    await session.execute(page.enable())
-    async with session.wait_for(page.LoadEventFired):
-        await session.execute(page.navigate(target_url))
-
-    # Extract the page title.
-    root_node = await session.execute(dom.get_document())
-    title_node_id = await session.execute(dom.query_selector(root_node.node_id,
-        'title'))
-    html = await session.execute(dom.get_outer_html(title_node_id))
-    print(html)
-```
-
-We'll go through this example bit by bit. First, it starts with a context
-manager:
-
-```python
-async with open_cdp(cdp_url) as conn:
-```
-
-This context manager opens a connection to the browser when the block is entered
-and closes the connection automatically when the block exits. Now we have a
-connection to the browser, but the browser has multiple targets that can be
-operated independently. For example, each browser tab is a separate target. In
-order to interact with one of them, we have to create a session for it.
-
-```python
-targets = await conn.execute(target.get_targets())
-target_id = targets[0].id
-```
-
-The first line here executes the `get_targets()` command in the browser. Note
-the form of the command: `await conn.execute(...)` will send a command to the
-browser, parse its response, and return a value (if any). The command is one of
-the methods in the PyCDP package. Trio CDP multiplexes commands and responses on
-a single connection, so we can send commands concurrently if we want, and the
-responses will be routed back to the correct task.
-
-In this case, the command is `target.get_targets()`, which returns a list of
-`TargetInfo` objects. We grab the first object and extract its `target_id`.
-
-```python
-session = await conn.open_session(target_id)
-```
-
-In order to connect to a target, we open a session based on the target ID.
-
-```python
-await session.execute(page.enable())
-async with session.wait_for(page.LoadEventFired):
-    await session.execute(page.navigate(target_url))
-```
-
-Here we use the session (remember, it corresponds to a tab in the browser) to
-navigate to the target URL. Just like the connection object, the session object
-has an `execute(...)` method that sends a command to the target, parses the
-response, and returns a value (if any).
-
-This snippet also introduces another concept: events. When we ask the browser to
-navigate to a URL, it acknowledges our request with a response, then starts the
-navigation process. How do we know when the page is actually loaded, though?
-Easy: the browser can send us an event!
-
-We first have to enable page-level events by calling `page.enable()`. Then we
-use `session.wait_for(...)` to wait for an event of the desired type. In this
-example, the script will suspend until it receives a `page.LoadEventFired`
-event. (After this block finishes executing, you can run `page.disable()` to
-turn off page-level events if you want to save some bandwidth and processing
-power, or you can use the context manager `async with session.page_enable(): ...`
-to automatically enable page-level events just for a specific block.)
-
-Note that we wait for the event inside an `async with` block, and we do this
-_before_ executing the command that will trigger this event. This order of
-operations may be surprising, but it avoids race conditions. If we executed a
-command and then tried to listen for an event, the browser might fire the event
-very quickly before we have had a chance to set up our event listener, and then
-we would miss it! The `async with` block sets up the listener before we run the
-command, so that no matter how fast the event fires, we are guaranteed to catch
-it.
-
-```python
-root_node = await session.execute(dom.get_document())
-title_node_id = await session.execute(
-    dom.query_selector(root_node.node_id, 'title'))
-html = await session.execute(dom.get_outer_html(title_node_id))
-print(html)
-```
-
-The last part of the script navigates the DOM to find the `<title>` element.
-First we get the document's root node, then we query for a CSS selector, then
-we get the outer HTML of the node. This snippet shows some new APIs, but the
-mechanics of sending commands and getting responses are the same as the previous
-snippets.
+    async with conn.open_session(target_id) as session:
 
-A more complete version of this example can be found in `examples/get_title.py`.
-There is also a screenshot example in `examples/screenshot.py`. The unit tests
-in `tests/` also provide more examples.
+        # Navigate to a website.
+        async with session.page_enable()
+        async with session.wait_for(page.LoadEventFired):
+            await session.execute(page.navigate(target_url))
 
-To run the examples, you need a Chrome binary in your system. You can get one like this:
-
-## Running Examples on MacOS
-
-**Terminal 1**
-
-This sets up the chrome browser in a specific version, and runs it in debug mode with Tor proxy for network traffic.
-
-```
-wget https://www.googleapis.com/download/storage/v1/b/chromium-browser-snapshots/o/Mac%2F678035%2Fchrome-mac.zip?generation=1563322360871926&alt=media
-unzip chrome-mac.zip && rm chrome-mac.zip
-./chrome-mac/Chromium.app/Contents/MacOS/Chromium --remote-debugging-port=9000
-> DevTools listening on ws://127.0.0.1:9000/devtools/browser/<DEV_SESSION_GUID>
-```
-
-**Terminal 2**
-
-This runs the example browser automation script on the instantiated browser window.
-
-```bash
-python examples/get_title.py ws://127.0.0.1:9000/devtools/browser/<DEV_SESSION_GUID> https://hyperiongray.com
-```
-
-## Running Examples on Linux
-
-**Terminal 1**
-
-This sets up the chrome browser in a specific version, and runs it in debug mode with Tor proxy for network traffic.
-
-```
-wget https://storage.googleapis.com/chromium-browser-snapshots/Linux_x64/678025/chrome-linux.zip
-unzip chrome-linux.zip && rm chrome-linux.zip
-./chrome-linux/chrome --remote-debugging-port=9000
-> DevTools listening on ws://127.0.0.1:9000/devtools/browser/<DEV_SESSION_GUID>
+        # Extract the page title.
+        root_node = await session.execute(dom.get_document())
+        title_node_id = await session.execute(dom.query_selector(root_node.node_id,
+            'title'))
+        html = await session.execute(dom.get_outer_html(title_node_id))
+        print(html)
 ```
 
-**Terminal 2**
-
-This runs the example browser automation script on the instantiated browser window.
-
-```bash
-python examples/get_title.py ws://127.0.0.1:9000/devtools/browser/<DEV_SESSION_GUID> https://hyperiongray.com
-```
-
-## Changelog
-
-### 0.5.0
-
-* **Backwards Compability Break:** Rename `open_cdp_connection()` to `open_cdp()`.
-* Fix `ConnectionClosed` bug.
-
-### 0.4.0
-
-* Add support for passing in a nursery. (Supports usage in Jupyter notebook.)
-
-### 0.3.0
-
-* New APIs for enabling DOM events and Page events.
-
-### 0.2.0
-
-* Restructure event listeners.
-
-### 0.1.0
-
-* Initial version
-
-<a href="https://www.hyperiongray.com/?pk_campaign=github&pk_kwd=trio-cdp"><img alt="define hyperion gray" width="500px" src="https://hyperiongray.s3.amazonaws.com/define-hg.svg"></a>
+This example code is explained [in the documentation](https://trio-cdp.readthedocs.io)
+and more example code can be found in the `examples/` directory of this repository.

docs/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/_build/doctrees/environment.pickle

@@ -0,0 +1,4 @@
+# Sphinx build info version 1
+# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
+config: 5306399389ef0e848bd6cdc19aec756e
+tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/_build/doctrees/index.doctree

@@ -0,0 +1,20 @@
+.. Trio CDP documentation master file, created by
+   sphinx-quickstart on Fri Mar 27 15:44:46 2020.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+Welcome to Trio CDP's documentation!
+====================================
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`