How to connect to SharePoint Online and and SharePoint 2013 2016 2019 on premises with app principal

Here is an instruction to create app based credentials:

1. Go to the appregnew.aspx page in your SharePoint Online tenant. For example, https://example.sharepoint.com/_layouts/15/appregnew.aspx.
2. On this page, click the Generate buttons next to the Client ID and Client Secret fields to generate their values.
3. Store the client ID and client secret securely as these credentials can be used to read or update all data in your SharePoint Online environment. You will also use them to configure the SharePoint Online connection in application.
4. Under Title, specify a title. For example, Python console. Under App Domain, specify localhost. Under Redirect URI, specify https://localhost.

Note: Sometimes, if you specify a actual domain, e.g. sharepoint.com domain in the App Domain and Redirect URI fields, instead of localhost, the error message An unexpected error has occurred might encounter. Check the appregnew.aspx page and make sure both fields include the proper localhost URI.

5. Click Create.

6. Go to the appinv.aspx page on the site collection. For example, https://example.sharepoint.com/_layouts/15/appinv.aspx to grant site-scoped permissions.

Note: If you prefer grant permissions on tenant level, visit tenant administration site instead, the URL must include -admin to access the tenant administration site, for example, https://example-admin.sharepoint.com/_layouts/15/appinv.aspx That operation requires a tenant administrator permissions

7. Specify your client ID in the App Id field and click Lookup to find your app. To grant permissions to the app, copy the XML below to the App’s permission request XML field:

<AppPermissionRequests AllowAppOnlyPolicy="true">
<AppPermissionRequest Scope="http://sharepoint/content/sitecollection" Right="FullControl" />
</AppPermissionRequests>

Note: For tenant level scope, permission request XML looks as follows:

<AppPermissionRequests AllowAppOnlyPolicy="true">
<AppPermissionRequest Scope="http://sharepoint/content/tenant" Right="FullControl" />
</AppPermissionRequests>

If you see the error message Sorry, only tenant administrators can add or give access to this app" and the Trust It button is disabled, you are not on the correct page for the tenant administration site. Check the appinv.aspx page URL and make sure it includes -admin.

8. Click Create.
9. On the confirmation dialog, click Trust It to grant the permissions.

As a proof-of-concept here is an example of running the following script in Jupiter Notebook

client_id = "--client id goes here--"
client_secret = "-- secret goes here--"
site_url = "https://example.sharepoint.com/"

from office365.runtime.auth.client_credential import ClientCredential
from office365.sharepoint.client_context import ClientContext


creds = ClientCredential(client_id, client_secret)
ctx = ClientContext(site_url).with_credentials(creds)
web = ctx.web
ctx.load(web)
ctx.execute_query()

Result

References

Granting access using SharePoint App-Only