Skip to content

Data Flow

halomore edited this page Feb 25, 2020 · 1 revision

Table of Contents

  1. Intro
  2. Summary

NOTE: Rough Draft

1. Intro

Understanding the data flow in PTS is quite complex. Overtime, I'll do my best to continue documenting the data flow here for you, so that you have a better understanding. What constrains the data flow is when a user who is novice and doesn't understand much about linux, but that's ok. We all started somewhere at one point.

2. Understanding UnionFS

Within all editions of PlexGuide, there is a certain path that is critical... and that is UnionFS. UnionFS is a path that merges multiple paths as one together. Well... you may ask... why does this need to occur in the first place. Ah, good question! A quick learner.

PlexGuide utilizes a program known as RClone. RClone provides the user that ability to move files locally and to other cloud drives. If you search for how to mount a Google Drive, you may tend to notice there is not a real solution and you'll stumble onto other GITHUB project like PlexGuide. To achieve this, PlexGuide mounts your Google Drive in very particular locations, including UnionFS. The paths that are merged are:

  • /mnt/move (writeable)
  • /mnt/gdrive (read only)
  • /mnt/tdrive (read only)
  • /mnt/unionfs (read only)

New users will attempt to send their data to /mnt/gdrive. A hard lesson to learn is that regardless of how you setup rclone, you are not able to directly move data to /mnt/gdrive. So wait? How does data get to gdrive then? First, a user will point their data to /mnt/unionfs ... yes.. yes... it's readonly. What makes UnionFS special is that it combined all your drives in the first place. You notice that /mnt/move is writeable correct? When data touches /mnt/unionfs, UnionFS says the following:

  • hey gdrive... can you write this? GDrive says... nope.
  • hey tdrive... can you write this? TDrive says... nope.
  • hey move... can you write this? Yes I can!

So now the data goes to /mnt/move. Awesome right? Well that was half the battle.

3. Data Movement

PlexGuide has two mechanism to assist you with the movement of your data! That is PTS Blitz and PTS Move.

PTS Move dead simple in that RClone is utilized to send data from /mnt/move to gdrive:/path. Basically, RClone is saying... hey data! I need you to move to gdrive:/path which is a cloud drive. RClone syncs everything and moves data from /mnt/move to gdrive:/path. You know what the great thing is? All your programs cannot tell that your data moved due to UnionFS combining all of the same paths.

PTS Blitz is a bit more complicated in that RClone is utilized to send data from /mnt/move, to /mnt/PTS/upload, then to tdrive:/path... which is a team drive. Why a team drive? Most users who utilize PTS Blitz do so to break the 750GB daily upload cap imposed by Google.

--- UNDER CONSTRUCTION STILL ---

Useful Starter Links

  1. Introduction
  2. G-Suite Signup
  3. Recommended NewsHosts
  4. Change Log

Prior Planning 101

  1. Recommended Pre-Reading
  2. Server - Storage Planning
  3. Usenet or BitTorrent
  4. PTS Editions
  5. PTS Folder Structure
  6. PTS-Repos
  7. Common Issues

Deploy & Config

PTS-Team
  1. SSH Server Access
  2. Install PTS
  3. WatchTower
  4. Remote Path Mappings

Data Transport
  1. PTS Clone
    1. Google OAuth Keys
    2. PTS Move
    3. PTS Blitz
    4. 2nd HD Option

Key Components

  1. Traefik
  2. Port Guard
  3. PTS Shield
  4. PTS Press
  5. Google Cloud - GCE
    1. Automated setup
  6. Hetzner Cloud
  7. PTS Fork
  8. Extra PTS Commands

Blitz App Info

  1. Accessing PTS Apps
  2. App Port Scheme

Tools & Services

Core Apps [Expand]
  1. Emby
  2. Jackett
  3. Jellyfin
  4. LazyLibrarian
  5. Lidarr
  6. Netdata
  7. NZBGet
  8. Ombi
  9. Plex
  10. Portainer
  11. qBittorrent
  12. Radarr
  13. Radarr4k
  14. RadarrHDR
  15. RuTorrent
  16. SABNZBD
  17. Sonarr
  18. Sonarr4k
  19. SonarrHDR
  20. Tautulli
  21. JDownloader2
Community Apps [Expand]
  1. AirSonic
  2. Bazarr
  3. Bitwarden
  4. BookSonic
  5. Calibre
  6. cAdvisor
  7. Cloud Commander
  8. Deluge
  9. DelugeVPN
  10. Dozzle
  11. Duplicati
  12. FlexGet
  13. Gazee
  14. Headphones
  15. Heimdall
  16. HomeAssistant
  17. jd2-openvpn
  18. Kitana
  19. Logarr
  20. MakeMKV
  21. MariaDB
  22. McMyadmin
  23. MEDUSA
  24. Mellow
  25. Miniflux
  26. Monitorr
  27. Muximux
  28. Mylar
  29. NextCloud
  30. NowShowing
  31. NZBHydra2
  32. Organizr
  33. plpp
  34. pyLoad
  35. qBittorrent
  36. QBittorrent VPN
  37. radarrsyncarr
  38. Resilio
  39. rflood-openvpn
  40. rutorrent-openvpn
  41. Shoko Anime Server
  42. SpeedTest
  43. SyncLounge
  44. Teamspeak3
  45. The Lounge
  46. Transmission
  47. Trakt.or
  48. Ubooquity
  49. Varken
  50. vnc-xfce
  51. ZeroTier

Misc Info & Articles

  1. Change Server Time
  2. Fail2Ban
  3. Custom Containers
  4. Deploying Varken, Influx, Telegraf, Grafana
  5. cloneclean commands
  6. Understanding htop
Scripts
  1. Scripting Area
    1. Auto Upload for Sonarr and Lidarr
    2. Auto Delete Log
    3. Plex Plugin DB Backup
    4. Plex fast BACKUP
    5. Delete All Unmonitored from Radarr
Clone this wiki locally