Merge remote-tracking branch 'upstream/master'

.github/workflows/python-app.yml

@@ -6,14 +6,17 @@ name: Python application
 on:
   push:
     branches: [ '**' ]
+  pull_request:
+    # The branches below must be a subset of the branches above
+    branches: [ "master" ]
 
 jobs:
   build:
     runs-on: ${{ matrix.os }}
 
     strategy:
       matrix:
-        python-version: [ 3.7, 3.8, 3.9, "3.10", "3.11" ]
+        python-version: [ 3.8, 3.9, "3.10", "3.11" ]
         os: [ ubuntu-latest, macos-latest, windows-latest ]
 
     steps:
@@ -25,10 +28,11 @@ jobs:
       - name: Lint with flake8
         run: |
           python -m pip install flake8
-          # stop the build if there are Python syntax errors or undefined names
-          flake8 . --count --select=E9,F63,F7,F82,F401 --show-source --statistics
-          # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
-          flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
+          flake8 . --max-line-length=127
+      - name: Verify sorted imports
+        run: |
+          python -m pip install isort
+          isort . -m HANGING_INDENT -l 120 --check-only
       - name: Test install
         run: |
           python -m pip install --upgrade pip

.github/workflows/python-publish.yml

@@ -21,11 +21,11 @@ jobs:
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
-        pip install setuptools wheel twine
+        pip install -U build setuptools wheel twine
     - name: Build and publish
       env:
         TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
         TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
       run: |
-        python setup.py sdist bdist_wheel
+        python -m build
         twine upload dist/*

CODE_OF_CONDUCT.md

@@ -60,7 +60,7 @@ representative at an online or offline event.
 
 Instances of abusive, harassing, or otherwise unacceptable behavior may be
 reported to the community leaders responsible for enforcement at
-https://gitter.im/pymobiledevice3/community.
+https://discord.gg/52mZGC3JXJ.
 All complaints will be reviewed and investigated promptly and fairly.
 
 All community leaders are obligated to respect the privacy and security of the

CONTRIBUTING.md

@@ -1,4 +1,4 @@
 If you would like to contribute, feel free to report issues, start new discussions, or create pull requests. You can
-also contact us on gitter:
+also contact us on Discord:
 
-https://gitter.im/pymobiledevice3/community
+https://discord.gg/52mZGC3JXJ

NEWS.md

@@ -0,0 +1,14 @@
+# News
+
+## 10/08/2023
+
+Features:
+
+- Trying to work with iOS >= 17.0 devices?
+  See [Working with developer tools (iOS >= 17.0)](#working-with-developer-tools-ios--170)
+
+## 28/06/2023
+
+API Changes:
+
+- Migrating from `pymobiledevice3<2.0.0`? See [this API change](README.md#python-api)

README.md

@@ -2,10 +2,13 @@
 [![Pypi version](https://img.shields.io/pypi/v/pymobiledevice3.svg)](https://pypi.org/project/pymobiledevice3/ "PyPi package")
 [![Downloads](https://static.pepy.tech/personalized-badge/pymobiledevice3?period=total&units=none&left_color=grey&right_color=blue&left_text=Downloads)](https://pepy.tech/project/pymobiledevice3)
 
+- [News](#news)
 - [Description](#description)
 - [Installation](#installation)
-    * [Lower iOS versions (<13)](#lower-ios-versions---13-)
+    * [OpenSSL libraries](#openssl-libraries)
 - [Usage](#usage)
+    * [Python API](#python-api)
+    * [Working with developer tools (iOS >= 17.0)](#working-with-developer-tools-ios--170)
     * [Example](#example)
 - [The bits and bytes](#the-bits-and-bytes)
     * [Lockdown services](#lockdown-services)
@@ -15,6 +18,11 @@
             - [Lockdown messages](#lockdown-messages)
             - [Instruments messages](#instruments-messages)
 - [Contributing](#contributing)
+- [Useful info](#Useful-info)
+
+# News
+
+See [NEWS](NEWS.md).
 
 # Description
 
@@ -27,6 +35,7 @@ both architecture and platform generic and is supported and tested on:
 
 Main features include:
 
+* Device discovery over bonjour
 * TCP port forwarding
 * Viewing syslog lines (including debug)
 * Profile management
@@ -74,11 +83,11 @@ eval "$(_PYMOBILEDEVICE3_COMPLETE=source_zsh pymobiledevice3)"
 eval "$(_PYMOBILEDEVICE3_COMPLETE=zsh_source pymobiledevice3)"
 ```
 
-## Lower iOS versions (<13)
+## OpenSSL libraries
 
-If you wish to use pymobiledevice3 with iOS versions lower than 13, Make sure to install `openssl`:
+Currently, openssl is explicitly required if using on older iOS version (<13).
 
-On MAC:
+On macOS:
 
 ```shell
 brew install openssl
@@ -110,6 +119,7 @@ Commands:
   amfi             amfi options
   apps             application options
   backup2          backup utils
+  bonjour          bonjour options
   companion        companion options
   crash            crash report options
   developer        developer options.
@@ -122,23 +132,75 @@ Commands:
   processes        processes cli
   profile          profile options
   provision        privision options
+  remote           remote options
   restore          restore options
   springboard      springboard options
   syslog           syslog options
-  usbmuxd          usbmuxd options
+  usbmux           usbmuxd options
   webinspector     webinspector options
 ```
 
-Or import the modules and use the API yourself:
+## Python API
+
+You could also import the modules and use the API yourself:
 
 ```python
-from pymobiledevice3.lockdown import LockdownClient
+from pymobiledevice3.remote.remote_service_discovery import RemoteServiceDiscoveryService
+from pymobiledevice3.lockdown import create_using_usbmux
 from pymobiledevice3.services.syslog import SyslogService
 
-lockdown = LockdownClient()
-for line in SyslogService(lockdown=lockdown).watch():
+# Connecting via usbmuxd
+lockdown = create_using_usbmux()
+for line in SyslogService(service_provider=lockdown).watch():
     # just print all syslog lines as is
     print(line)
+
+# Or via remoted (iOS>=17)
+# First, create a tunnel using:
+#     $ sudo pymobiledevice3 remote start-quic-tunnel
+# You can of course implement it yourself by copying the same pieces of code from:
+#     https://github.com/doronz88/pymobiledevice3/blob/master/pymobiledevice3/cli/remote.py#L68
+# Now you can simply connect to the created tunnel's host and port
+host = 'fded:c26b:3d2f::1'  # randomized
+port = 65177  # randomized
+with RemoteServiceDiscoveryService((host, port)) as rsd:
+    for line in SyslogService(service_provider=rsd).watch():
+        # just print all syslog lines as is
+        print(line)
+```
+
+## Working with developer tools (iOS >= 17.0)
+
+> **NOTE:** Currently, this is only supported on macOS
+
+Starting at iOS 17.0, Apple introduced the new CoreDevice framework to work with iOS devices. This framework relies on
+the [RemoteXPC](misc/RemoteXPC.md) protocol. In order to communicate with the developer services you'll be required to
+first create [trusted tunnel](misc/RemoteXPC.md#trusted-tunnel) as follows:
+
+```shell
+sudo python3 -m pymobiledevice3 remote start-quic-tunnel
+```
+
+The root permissions are required since this will create a new TUN/TAP device which is a high privilege operation.
+The output should be something similar to:
+
+```
+Interface: utun6
+RSD Address: fd7b:e5b:6f53::1
+RSD Port: 64337
+Use the follow connection option:
+--rsd fd7b:e5b:6f53::1 64337
+```
+
+Now, (almost) all of pymobiledevice3 accept an additional `--rsd` option for connecting to the service over this new
+tunnel. You can now try to execute any of them as follows:
+
+```shell
+# Accessing the DVT services
+python3 -m pymobiledevice3 developer dvt ls / --rsd fd7b:e5b:6f53::1 64337
+
+# Or any of the "normal" ones
+python3 -m pymobiledevice3 syslog live --rsd fd7b:e5b:6f53::1 64337
 ```
 
 ## Example
@@ -149,7 +211,9 @@ https://terminalizer.com/view/18920b405193
 There is A LOT you may do on the device using `pymobiledevice3`. This is just a TL;DR of some common operations:
 
 * Listing connected devices:
-    * `pymobiledevice3 list-devices`
+    * `pymobiledevice3 usbmux list`
+* Discover network devices using bonjour:
+    * `pymobiledevice3 bonjour browse`
 * View all syslog lines (including debug messages):
     * `pymobiledevice3 syslog live`
 * Filter out only messages containing the word "SpringBoard":
@@ -175,18 +239,20 @@ There is A LOT you may do on the device using `pymobiledevice3`. This is just a
         * `pymobiledevice3 webinspector opened-tabs`
     * The following will require also the Remote Automation feature to be turned on:
         * Get interactive JavaScript shell on new remote automation tab:
-            * `pymobiledevice3 webinspector js_shell --automation` 
+            * `pymobiledevice3 webinspector js_shell --automation`
         * Launch an automation session to view a given URL:
             * `pymobiledevice3 webinspector launch URL`
         * Get a a selenium-like shell:
             * `pymobiledevice3 webinspector shell`
-* Mount DeveloperDiskImage:
-    * `pymobiledevice3 mounter mount`
+* Mount DeveloperDiskImage (On iOS>=17.0, each command will require an additional `--rsd` option):
+    * `pymobiledevice3 mounter auto-mount`
     * The following will assume the DeveloperDiskImage is already mounted:
         * Simulate an `x y` location:
             * `pymobiledevice3 developer simulate-location set x y`
+            * Or the following for iOS>=17.0:
+                * `pymobiledevice3 developer dvt simulate-location set --rsd HOST PORT -- x y`
         * Taking a screenshot from the device:
-            * `pymobiledevice3 developer screenshot /path/to/screen.png`
+            * `pymobiledevice3 developer dvt screenshot /path/to/screen.png`
         * View detailed process list (including ppid, uid, guid, sandboxed, etc...):
             * `pymobiledevice3 developer dvt sysmon process single`
         * Sniffing oslog:
@@ -381,3 +447,7 @@ return_value, auxiliary = developer.recv_plist()
 # Contributing
 
 See [CONTRIBUTING](CONTRIBUTING.md).
+
+# Useful info
+
+Please see [misc](misc)