Local deployment

[Tomaubier]

As a PhD student relying heavily on Obsidian, I'm looking for a way to share some of my notes with my lab colleagues.
This project seems to fit my needs perfectly however I'm wondering wether it would be possible to perform the deployment on a local server on a docker container for example?
Best regards
Tom

[oleeskild]

I don't know your skill level when it comes to node and docker, so excuse me if I'm either over or under explaining this. Let me know if you have any questios, or something isn't clear.

I threw something together you could use to run this locally:

1. Clone your digital garden repo to your computer
2. run npm install express --save to install express as a dependency.
3. Create a new file in the root folder called app.js, and paste the following code:

const express = require('express')
const app = express()
const port = 8080; 

const searchHandler = require('./netlify/functions/search/search.js').handler;

app.get('/api/search', async (req, res) => {
  //Mock the netlify event
  let event = {queryStringParameters: req.query};

  let response = await searchHandler(event);
  res.status(response.statusCode).send(response.body);
})

app.use(express.static('dist'))

app.get('*', (req, res)=>{
  res.redirect("/404")
})

app.listen(port, () => {
  console.log(`Digital garden running on port ${port}`)
})

4. Run npm install
5. Run npm run build
6. Run node app.js

The site should now be available at localhost:8080.

You can then use something like pm2 to run it on your server, and use a reverse nginx proxy with lets encrypt to expose it on port 443 with https.

The one thing it is missing is automatically pulling the latest changes when you publish a note.

I will make a docker file available that does all this sometime in the future, when I find the time, so everybody has this option easily available.

Hopefully this is enough to get you started.

If you manage to actually create a docker file for this, I would be very happy to accept a PR 😄

[davidkopp]

Alternative Dockerfile using Nginx:

FROM node:18 as base
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .

FROM base as builder
WORKDIR /usr/src/app
RUN npm run build

FROM nginx:1.24
COPY ./nginx/nginx.conf /etc/nginx/conf.d/default.conf
COPY --from=builder /usr/src/app/dist/ /usr/share/nginx/html/
EXPOSE 8080

Nginx config file (nginx/nginx.conf):

server { 

    listen 8080;
    server_name localhost;

    error_page 404 /404/index.html;
    charset UTF-8;

    root /usr/share/nginx/html;

    location / {
        index index.html
        try_files $uri $uri/ =404;
    }
}

@oleeskild Do you know of any drawback of using Nginx instead of Express?
What is the purpose of the API endpoint /api/search in your app.js file?

[davidkopp]

Ok, finally, my third version and probably also the last one: Dockerfile for only building and exporting.
I am now using this approach because I already have Nginx in use as a reverse proxy for other services on the same server, which now simply delivers the files as well. The purpose of the Dockerfile is that the complete build process takes place in a container and no dependencies need to be installed on the host system.

Dockerfile (Dockerfile.export):

FROM node:18 as base
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .

FROM base as builder
WORKDIR /usr/src/app
RUN npm run build

FROM scratch AS export
COPY --from=builder /usr/src/app/dist /

Build command that places the resulting dist directory in a dist directory on the host system:

podman build -f Dockerfile.export --output dist .

[wchorski]

Any advice on how to process markdown files in the vault before it hits the 11ty app repo?

• Can we use the Obsidian Plugin outside of Obsidian?
• Do we have to use GitHub for this process?

[davidkopp]

@wchorski The Obsidian Digital Garden Plugin is only available for Obsidian and currently only supports GitHub.

To make some changes to your markdown files before they gets processed by 11ty you can use the feature "Custom Filters": https://dg-docs.ole.dev/getting-started/05-other-settings/#custom-filters
For example, I'm using custom filters to remove links to local files.

[janfromm]

Hi!
In the root folder of my digital garden repo there is no subfolder named netlify.
When trying to run app.js, I'm receiving an error:

node:internal/modules/cjs/loader:1147
  throw err;
  ^

Error: Cannot find module './netlify/functions/search/search.js'
Require stack:
- /[...]/self-host/app.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1144:15)
    at Module._load (node:internal/modules/cjs/loader:985:27)
    at Module.require (node:internal/modules/cjs/loader:1235:19)
    at require (node:internal/modules/helpers:176:18)
    at Object.<anonymous> (/[...]/self-host/app.js:5:23)
    at Module._compile (node:internal/modules/cjs/loader:1376:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1435:10)
    at Module.load (node:internal/modules/cjs/loader:1207:32)
    at Module._load (node:internal/modules/cjs/loader:1023:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:135:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '[...]/self-host/app.js' ]
}

Node.js v20.11.0

What's my mistake?
Thanks for being pacient, I'm just getting started with node.js

Jan

[naquad]

Could you please clarify, who performs the actual Markdown -> HTML rendering?
The server script does not handle the notes files in any way. Is it some Vercel/Netlify magic?

Thank you.

[dayne]

@naquad the answer is Eleventy. Under the hood this plugin is taking the markdown files selected for publishing, bundling them up into a github repository, that is then rendered into a static website using the 11ty static site generator. The rendering of the github repo has to be "run" somewhere, and the default pathway documented is to hook up the github repo to vercel for the rendering+hosting dance. It could be done 100% on GitHub + GitHub Actions if you want to skip that part.

[naquad]

Thank you. I somehow thought it was UI-related rather than the whole rendering.

[ninijay]

can you elaborate how that could work with GitHub + GitHub Actions? I'm unable to find a guide

[rubenrivera]

@janfromm

The template repository has changed since @oleeskild posted his answer. One significant change is that the default hosting platform was Netlify, but it was swapped to Vercel. I'm not sure if this swapping caused the removal of the Netlify directory, but new repositories don't include it.

Try commenting out or removing the statements that use searchHandler

const express = require('express')
const app = express()
const port = 8080; 
/**
const searchHandler = require('./netlify/functions/search/search.js').handler;

app.get('/api/search', async (req, res) => {
  //Mock the netlify event
  let event = {queryStringParameters: req.query};

  let response = await searchHandler(event);
  res.status(response.statusCode).send(response.body);
})
*/

app.use(express.static('dist'))

app.get('*', (req, res)=>{
  res.redirect("/404")
})

app.listen(port, () => {
  console.log(`Digital garden running on port ${port}`)
})

Hi! In the root folder of my digital garden repo there is no subfolder named netlify. When trying to run app.js, I'm receiving an error:

node:internal/modules/cjs/loader:1147
  throw err;
  ^

Error: Cannot find module './netlify/functions/search/search.js'
Require stack:
- /[...]/self-host/app.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1144:15)
    at Module._load (node:internal/modules/cjs/loader:985:27)
    at Module.require (node:internal/modules/cjs/loader:1235:19)
    at require (node:internal/modules/helpers:176:18)
    at Object.<anonymous> (/[...]/self-host/app.js:5:23)
    at Module._compile (node:internal/modules/cjs/loader:1376:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1435:10)
    at Module.load (node:internal/modules/cjs/loader:1207:32)
    at Module._load (node:internal/modules/cjs/loader:1023:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:135:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '[...]/self-host/app.js' ]
}

Node.js v20.11.0

What's my mistake? Thanks for being pacient, I'm just getting started with node.js

Jan

[janfromm]

@rubenrivera

Hello,
thank you for your explanation, which I unfortunately only noticed recently.

I ended up doing the following:

1.) I created a Netlify project as described in the Digital-Garden-Docs (https://dg-docs.ole.dev/advanced/hosting-alternatives/).

2.) I asked Chat-GPT to create a shell script that automatically pulls the changes from my Digital Garden Github repo to my own server and then creates the "dist" directory with static html files via the Netlify CLI (https://docs.netlify.com/cli/get-started/).

3.) I use the following Express server to provide the Dist directory:

`const express = require('express');
const path = require('path');

const app = express();
const PORT = process.env.PORT || 3000;

app.use(express.static(path.join(__dirname, 'dist')));

app.listen(PORT, () => {
console.log(Server läuft auf http://localhost:${PORT});
});`

I'm relatively happy with this system because it's been fast and reliable so far. The Express server also gives me great opportunities to combine the static blog with a web app and implement additional functions.

However, it really bothers me that I don't know how Netlify converts the Digital Garden repo into static HTML, because my entire blog depends on this (closed source, third-party) program. If it ever stops working like it does now, I wouldn't know what to do.

Does anyone here know a way to do the conversion locally without relying on a third-party service?

[mounta11n]

I think the most convenient way is to deploy your digital garden in the very same git repo, but as a new branch and as a github page. this way you dont need netlify, vercel, a proxy server, docker or anything else. just one git repo and nothing more.
you have to name the forked repo your_user_name.github.io. here is an example where i did exactly this (with my other github account):

https://github.com/geschenkwald/geschenkwald.github.io

so the belongig digital-garden is (by github design, as mentioned, username.github.io): geschenkwald.github.io/

or, as you can see in my modified build.yaml file, you can set a custom CNAME, which I set to: cognithink.de

Additionally what is really really cool, you can integrate giscus in your blog/digital garden by utilizing the discussion function of the same repo as well. to get this working you have to edit the template files slightly (which are src/site/_includes/layouts/index.njk and note.njk). so in my case for example i basically added the following:

  <!-- ... -->
      {% endfor %}
  {{ content | hideDataview | taggify | link | safe}}
  
  <!-- comments section placeholder -->
  <div class="giscus"></div>
  
  <!-- giscus-script -->
  <script src="https://giscus.app/client.js"
          data-repo="geschenkwald/geschenkwald.github.io"
          data-repo-id="R_kgDOLBDuTA"
          data-category="Announcements"
          data-category-id="DIC_kwDOLBDuTM4Cf6VU"
          data-mapping="title"
          data-strict="0"
          data-reactions-enabled="0"
          data-emit-metadata="0"
          data-input-position="top"
          data-theme="light"
          data-lang="de"
          data-loading="lazy"
          crossorigin="anonymous"
          async>
  </script>
  
  {% for imp in dynamics.common.afterContent %}
    <!-- ... -->

And as you can see there is now a perfectly working comments sections below each of my blog articles and since i added myself (this account) as a collaborator in the settings of the geschenkwald.github.io repo, i can directly moderate, edit, delete comments in the discussion section without even having to logout and login with the other account.

@oleeskild if you are interested in this approach, i could make a new branch with a clean code and a pull request (or two, one for gh-page deployment and one for the giscus feature). additionally for giscus we could add an integration in the obsidian plugin, for example an option which the user can opt-in (like the file-tree-sidebar enable/disable etc).

[juanjesusov]

Hi! @mounta11n I consider that your proposal is just what I was looking for, I have done the steps you mention in this repo. Then from obsidian I have sent the commits to generate the web, however the link associated to github pages displays the following screen which is basically the README file.

I would like to ask if you know what step I am missing, I assume it is some configuration to define the path to the files I have generated with obsidian, however I have not been able to find the solution.

[pedroporo]

the github pages sould look like this

If you want you can look at my repo in case it helps you

[carakesa]

This is brilliant and basically what I was hoping to do when I first started wanting to do a digital garden and the whole Vercel setup (and it's associated bits and services) is really what turned me away. I'll have to check this out and see if I can put it in practice on my end

…

On Thu, Jun 6, 2024, 6:34 PM Yazan Agha-Schrader ***@***.***> wrote: I think the most convenient way is to deploy your digital garden in the very same git repo, but as a new branch and as a github page. this way you dont need netlify, vercel, a proxy server, docker or anything else. just one git repo and nothing more. you have to name the forked repo your_user_name.github.io. here is an example where i did exactly this (with my other github account): https://github.com/geschenkwald/geschenkwald.github.io so the belongig digital-garden is (by github design, as mentioned, username.github.io): geschenkwald.github.io/ or, as you can see in my modified build.yaml file, you can set a custom CNAME, which I set to: cognithink.de Additionally what is really really cool, you can integrate giscus <https://giscus.app/> in your blog/digital garden by utilizing the discussion function of the same repo as well. to get this working you have to edit the template files slightly (which are src/site/_includes/layouts/index.njk and note.njk). so in my case for example i basically added the following: <!-- ... --> {% endfor %} {{ content | hideDataview | taggify | link | safe}} <!-- comments section placeholder --> <div class="giscus"></div> <!-- giscus-script --> <script src="https://giscus.app/client.js" data-repo="geschenkwald/geschenkwald.github.io" data-repo-id="R_kgDOLBDuTA" data-category="Announcements" data-category-id="DIC_kwDOLBDuTM4Cf6VU" data-mapping="title" data-strict="0" data-reactions-enabled="0" data-emit-metadata="0" data-input-position="top" data-theme="light" data-lang="de" data-loading="lazy" crossorigin="anonymous" async> </script> {% for imp in dynamics.common.afterContent %} <!-- ... --> And as you can see there is now a perfectly working comments sections below each of my blog articles and since i added myself (this account) as a collaborator in the settings of the geschenkwald.github.io repo, i can directly moderate, edit, delete comments in the discussion section without even having to logout and login with the other account. @oleeskild <https://github.com/oleeskild> if you are interested in this approach, i could make a new branch with a clean code and a pull request (or two, one for gh-page deployment and one for the giscus feature). additionally for giscus we could add an integration in the obsidian plugin, for example an option which the user can opt-in (like the file-tree-sidebar enable/disable etc). — Reply to this email directly, view it on GitHub <#160 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AM3K3S6Y4OAF3GMDDXG2NRTZGDPWHAVCNFSM6AAAAAATZPS2C2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TMOJUGU2TM> . You are receiving this because you are subscribed to this thread.Message ID: <oleeskild/obsidian-digital-garden/repo-discussions/160/comments/9694556@ github.com>

[mounta11n]

le

[Ryj3l]

### Then write this Dockerfile

FROM node:18 AS build
WORKDIR /usr/src/app

# Copy package files to install dependencies
COPY package*.json ./

# Install all dependencies (including devDependencies)
RUN npm install

# Copy the rest of the application code
COPY . .

# (Optional) Build the application if there's a build step, but make sure to skip it gracefully if it's missing
RUN if [ -f package.json ] && grep -q "build" package.json; then npm run build; else echo "No build step"; fi

# Stage 2: Production environment
FROM node:18-slim

# Set environment variables for production
ENV NODE_ENV=production

# Set the working directory
WORKDIR /usr/src/app

# Copy the package files (for production dependencies)
COPY package*.json ./

# Install only production dependencies
RUN npm ci --only=production

# Copy the application code and built files from the build stage
COPY --from=build /usr/src/app .

# Expose the port that the app will run on
EXPOSE 8080

# Start the application
CMD ["node", "app.js"]

Nginx.conf

    listen 80;
    server_name yourdomain.com www.yourdomain.com;

    location /blog/ {
        proxy_pass http://localhost:8080/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl;
    server_name yourdomain.com www.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    location /blog/ {
        proxy_pass http://localhost:8080/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

At this point as you know all you need is a docker-compose file. Build your stack with nginx.

[scrapix]

Hello, can someone please share this project as consumable docker image?
I'm not able to build on osx :?

[dangehub]

Hello, can someone please share this project as consumable docker image? I'm not able to build on osx :?

I tried building one, but I only tested it in wsl.

https://hub.docker.com/r/qudange/obsidian-digital-garden