deploy updates

_sources/compute_server.rst.txt

@@ -276,6 +276,42 @@ When you edit files via CoCalc graphical interface, they are usually synced betw
     Sync Files Button for a Compute Server
 
 
+Compute Server VPN
+------------------
+
+In addition to Internet access using standard tools, your compute servers *in the same project* can communicate with each other over encrypted VPN. You can refer to your servers either using ``compute-server-nnnn`` names or, if you have configured a subdomain ``https://server_name.cocalc.cloud``, using ``server_name``. Here is quick example of a terminal session on a compute server::
+
+    (compute-server-1268) ~$ more /etc/hosts
+    ...
+    ### COCALC VPN -- EVERYTHING BELOW IS AUTOGENERATED -- DO NOT EDIT
+    10.11.12.242 compute-server-1268
+    10.202.0.87 internal-1268
+    34.162.173.49 external-1268
+    10.11.211.165 compute-server-4300
+    10.11.211.165 fire
+    10.0.3.110 internal-4300
+    149.36.0.137 external-4300
+    (compute-server-1268) ~$ sudo apt install -y iputils-ping
+    ...
+    (compute-server-1268) ~$ ping compute-server-4300
+    PING compute-server-4300 (10.11.211.165) 56(84) bytes of data.
+    64 bytes from compute-server-4300 (10.11.211.165): icmp_seq=1 ttl=64 time=231 ms
+    64 bytes from compute-server-4300 (10.11.211.165): icmp_seq=2 ttl=64 time=114 ms
+    ...
+    (compute-server-1268) ~$ ping fire               
+    PING fire (10.11.211.165) 56(84) bytes of data.
+    64 bytes from compute-server-4300 (10.11.211.165): icmp_seq=1 ttl=64 time=114 ms
+    64 bytes from compute-server-4300 (10.11.211.165): icmp_seq=2 ttl=64 time=114 ms
+    ...
+
+All traffic is fully encrypted and all ports are open, so you have complete freedom in configuring services that run over this VPN. IP addresses of the form ``10.11.x.x`` remain fixed for the lifetime of the compute server.
+
+Current limitations (that may be removed in the future):
+
+- the CoCalc project itself is *not* on this VPN
+- you can connect from any to any server, mixing different clouds and on prem, *except* two on prem servers
+- when you create a new server or configure a domain name for an existing one, you may need to *wait a few seconds* before the new name will become resolvable
+
 .. _choosing_a_cloud:
 
 Choosing a Cloud

compute_server.html

@@ -74,6 +74,7 @@
 <li class="toctree-l3"><a class="reference internal" href="#becoming-root-and-port-forwarding">Becoming root and Port Forwarding</a></li>
 <li class="toctree-l3"><a class="reference internal" href="#billing-for-a-compute-server">Billing for a Compute Server</a></li>
 <li class="toctree-l3"><a class="reference internal" href="#compute-server-filesystem">Compute Server Filesystem</a></li>
+<li class="toctree-l3"><a class="reference internal" href="#compute-server-vpn">Compute Server VPN</a></li>
 <li class="toctree-l3"><a class="reference internal" href="#choosing-a-cloud">Choosing a Cloud</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="#cost">Cost</a></li>
 <li class="toctree-l4"><a class="reference internal" href="#availability">Availability</a></li>
@@ -148,18 +149,19 @@
 <li><p><a class="reference internal" href="#becoming-root-and-port-forwarding" id="id26">Becoming root and Port Forwarding</a></p></li>
 <li><p><a class="reference internal" href="#billing-for-a-compute-server" id="id27">Billing for a Compute Server</a></p></li>
 <li><p><a class="reference internal" href="#compute-server-filesystem" id="id28">Compute Server Filesystem</a></p></li>
-<li><p><a class="reference internal" href="#choosing-a-cloud" id="id29">Choosing a Cloud</a></p>
+<li><p><a class="reference internal" href="#compute-server-vpn" id="id29">Compute Server VPN</a></p></li>
+<li><p><a class="reference internal" href="#choosing-a-cloud" id="id30">Choosing a Cloud</a></p>
 <ul>
-<li><p><a class="reference internal" href="#cost" id="id30">Cost</a></p></li>
-<li><p><a class="reference internal" href="#availability" id="id31">Availability</a></p></li>
-<li><p><a class="reference internal" href="#geographic-location" id="id32">Geographic Location</a></p></li>
-<li><p><a class="reference internal" href="#special-considerations" id="id33">Special Considerations</a></p></li>
-<li><p><a class="reference internal" href="#google" id="id34">Google</a></p></li>
-<li><p><a class="reference internal" href="#hyperstack" id="id35">Hyperstack</a></p></li>
-<li><p><a class="reference internal" href="#on-prem" id="id36">On Prem</a></p></li>
+<li><p><a class="reference internal" href="#cost" id="id31">Cost</a></p></li>
+<li><p><a class="reference internal" href="#availability" id="id32">Availability</a></p></li>
+<li><p><a class="reference internal" href="#geographic-location" id="id33">Geographic Location</a></p></li>
+<li><p><a class="reference internal" href="#special-considerations" id="id34">Special Considerations</a></p></li>
+<li><p><a class="reference internal" href="#google" id="id35">Google</a></p></li>
+<li><p><a class="reference internal" href="#hyperstack" id="id36">Hyperstack</a></p></li>
+<li><p><a class="reference internal" href="#on-prem" id="id37">On Prem</a></p></li>
 </ul>
 </li>
-<li><p><a class="reference internal" href="#teaching-with-compute-servers" id="id37">Teaching with Compute Servers</a></p></li>
+<li><p><a class="reference internal" href="#teaching-with-compute-servers" id="id38">Teaching with Compute Servers</a></p></li>
 </ul>
 </nav>
 <section id="cocalc-s-most-unique-offering">
@@ -396,11 +398,46 @@ <h2><a class="toc-backref" href="#id27" role="doc-backlink">Billing for a Comput
 </figcaption>
 </figure>
 </section>
+<section id="compute-server-vpn">
+<h2><a class="toc-backref" href="#id29" role="doc-backlink">Compute Server VPN</a><a class="headerlink" href="#compute-server-vpn" title="Link to this heading"></a></h2>
+<p>In addition to Internet access using standard tools, your compute servers <em>in the same project</em> can communicate with each other over encrypted VPN. You can refer to your servers either using <code class="docutils literal notranslate"><span class="pre">compute-server-nnnn</span></code> names or, if you have configured a subdomain <code class="docutils literal notranslate"><span class="pre">https://server_name.cocalc.cloud</span></code>, using <code class="docutils literal notranslate"><span class="pre">server_name</span></code>. Here is quick example of a terminal session on a compute server:</p>
+<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>(compute-server-1268) ~$ more /etc/hosts
+...
+### COCALC VPN -- EVERYTHING BELOW IS AUTOGENERATED -- DO NOT EDIT
+10.11.12.242 compute-server-1268
+10.202.0.87 internal-1268
+34.162.173.49 external-1268
+10.11.211.165 compute-server-4300
+10.11.211.165 fire
+10.0.3.110 internal-4300
+149.36.0.137 external-4300
+(compute-server-1268) ~$ sudo apt install -y iputils-ping
+...
+(compute-server-1268) ~$ ping compute-server-4300
+PING compute-server-4300 (10.11.211.165) 56(84) bytes of data.
+64 bytes from compute-server-4300 (10.11.211.165): icmp_seq=1 ttl=64 time=231 ms
+64 bytes from compute-server-4300 (10.11.211.165): icmp_seq=2 ttl=64 time=114 ms
+...
+(compute-server-1268) ~$ ping fire
+PING fire (10.11.211.165) 56(84) bytes of data.
+64 bytes from compute-server-4300 (10.11.211.165): icmp_seq=1 ttl=64 time=114 ms
+64 bytes from compute-server-4300 (10.11.211.165): icmp_seq=2 ttl=64 time=114 ms
+...
+</pre></div>
+</div>
+<p>All traffic is fully encrypted and all ports are open, so you have complete freedom in configuring services that run over this VPN. IP addresses of the form <code class="docutils literal notranslate"><span class="pre">10.11.x.x</span></code> remain fixed for the lifetime of the compute server.</p>
+<p>Current limitations (that may be removed in the future):</p>
+<ul class="simple">
+<li><p>the CoCalc project itself is <em>not</em> on this VPN</p></li>
+<li><p>you can connect from any to any server, mixing different clouds and on prem, <em>except</em> two on prem servers</p></li>
+<li><p>when you create a new server or configure a domain name for an existing one, you may need to <em>wait a few seconds</em> before the new name will become resolvable</p></li>
+</ul>
+</section>
 <section id="choosing-a-cloud">
-<span id="id3"></span><h2><a class="toc-backref" href="#id29" role="doc-backlink">Choosing a Cloud</a><a class="headerlink" href="#choosing-a-cloud" title="Link to this heading"></a></h2>
+<span id="id3"></span><h2><a class="toc-backref" href="#id30" role="doc-backlink">Choosing a Cloud</a><a class="headerlink" href="#choosing-a-cloud" title="Link to this heading"></a></h2>
 <p>It is impossible to choose “the best cloud” for compute servers overall, which is why CoCalc integrates with many of them! For each particular use case one may be more or less preferable - sometimes it may be obvious, sometimes it requires experimentation, and sometimes the difference is inconsequential. The purpose of this section is to highlight some points to consider when choosing a cloud, hopefully it will make this task easier for you.</p>
 <section id="cost">
-<h3><a class="toc-backref" href="#id30" role="doc-backlink">Cost</a><a class="headerlink" href="#cost" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id31" role="doc-backlink">Cost</a><a class="headerlink" href="#cost" title="Link to this heading"></a></h3>
 <p>This may be the most obvious criterion, but while we strive to be as transparent as possible with the cost of running a compute server, do keep in mind also that:</p>
 <ul class="simple">
 <li><p>there is a cost for the disk space when the server is off;</p></li>
@@ -409,11 +446,11 @@ <h3><a class="toc-backref" href="#id30" role="doc-backlink">Cost</a><a class="he
 </ul>
 </section>
 <section id="availability">
-<h3><a class="toc-backref" href="#id31" role="doc-backlink">Availability</a><a class="headerlink" href="#availability" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id32" role="doc-backlink">Availability</a><a class="headerlink" href="#availability" title="Link to this heading"></a></h3>
 <p>This one may be less obvious than the cost, but it is even more important - the price of a compute server does not matter if you cannot start it! In most cases once you managed to start a compute server, you can continue using it until you explicitly turn it off. However, there is no guarantee that you will be able to start the same machine again in a few days - other users may utilize all resources.</p>
 </section>
 <section id="geographic-location">
-<h3><a class="toc-backref" href="#id32" role="doc-backlink">Geographic Location</a><a class="headerlink" href="#geographic-location" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id33" role="doc-backlink">Geographic Location</a><a class="headerlink" href="#geographic-location" title="Link to this heading"></a></h3>
 <p>Your organization may impose strict requirements on where your data are stored and processed. Other factors to consider:</p>
 <ul class="simple">
 <li><p>latency for interactive work: a server close to you may be best if you directly use web services started on it, while <code class="docutils literal notranslate"><span class="pre">us-east1-d</span></code> may be optimal to use CoCalc editors;</p></li>
@@ -422,35 +459,35 @@ <h3><a class="toc-backref" href="#id32" role="doc-backlink">Geographic Location<
 </ul>
 </section>
 <section id="special-considerations">
-<h3><a class="toc-backref" href="#id33" role="doc-backlink">Special Considerations</a><a class="headerlink" href="#special-considerations" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id34" role="doc-backlink">Special Considerations</a><a class="headerlink" href="#special-considerations" title="Link to this heading"></a></h3>
 <p>You may want to use a data centre satisfying particular requirements on sustainability, security, etc. We try to expose as much information about our cloud providers as possible to help our users to make an informed choice.</p>
 <p>Let’s take a look now at some of the unique benefits of our cloud providers!</p>
 </section>
 <section id="google">
-<h3><a class="toc-backref" href="#id34" role="doc-backlink">Google</a><a class="headerlink" href="#google" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id35" role="doc-backlink">Google</a><a class="headerlink" href="#google" title="Link to this heading"></a></h3>
 <ul class="simple">
 <li><p>Vast selection of varied configurations!</p></li>
 <li><p>For computationally intensive tasks, do take a look at higher end machines. We had customers with very strict requirements on interconnect network between cluster nodes, but it turned out that Google had single nodes with enough vCPUs, eliminating the need for a cluster setup!</p></li>
 <li><p>Spot instances offer up to 10x lower prices, if your work can deal with potential interruptions!</p></li>
 </ul>
 </section>
 <section id="hyperstack">
-<h3><a class="toc-backref" href="#id35" role="doc-backlink">Hyperstack</a><a class="headerlink" href="#hyperstack" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id36" role="doc-backlink">Hyperstack</a><a class="headerlink" href="#hyperstack" title="Link to this heading"></a></h3>
 <ul class="simple">
 <li><p>Apart from cost and speed, Hyperstack prioritizes sustainability of their data centres. They are 100% powered by hydro-electricity and are located in Canada and Norway to benefit from lower ambient temperature for air cooling.</p></li>
 <li><p>You can see how many GPUs are available on Hyperstack at any given time and estimate your chances of being able to get what you need in the future.</p></li>
 </ul>
 </section>
 <section id="on-prem">
-<h3><a class="toc-backref" href="#id36" role="doc-backlink">On Prem</a><a class="headerlink" href="#on-prem" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id37" role="doc-backlink">On Prem</a><a class="headerlink" href="#on-prem" title="Link to this heading"></a></h3>
 <ul class="simple">
 <li><p>You are in full control of the machine you are using.</p></li>
 <li><p>At the moment On Prem compute servers are free, the intended cost when their development is finished is about 1/3 of a similar spot instance on Google. If you <em>already own</em> sufficient compute resources, this may be the most cost-effective option to benefit from the cloud and collaboration as well!</p></li>
 </ul>
 </section>
 </section>
 <section id="teaching-with-compute-servers">
-<span id="id4"></span><h2><a class="toc-backref" href="#id37" role="doc-backlink">Teaching with Compute Servers</a><a class="headerlink" href="#teaching-with-compute-servers" title="Link to this heading"></a></h2>
+<span id="id4"></span><h2><a class="toc-backref" href="#id38" role="doc-backlink">Teaching with Compute Servers</a><a class="headerlink" href="#teaching-with-compute-servers" title="Link to this heading"></a></h2>
 <p>Compute servers are a great option to let your students or workshop participants use GPUs or powerful compute resources! Some important points to consider ahead of the course start:</p>
 <ul>
 <li><p>What configuration do you need for your students?</p></li>

service_options.html

@@ -156,6 +156,7 @@ <h1>Service Options<a class="headerlink" href="#service-options" title="Link to
 <li class="toctree-l2"><a class="reference internal" href="compute_server.html#becoming-root-and-port-forwarding">Becoming root and Port Forwarding</a></li>
 <li class="toctree-l2"><a class="reference internal" href="compute_server.html#billing-for-a-compute-server">Billing for a Compute Server</a></li>
 <li class="toctree-l2"><a class="reference internal" href="compute_server.html#compute-server-filesystem">Compute Server Filesystem</a></li>
+<li class="toctree-l2"><a class="reference internal" href="compute_server.html#compute-server-vpn">Compute Server VPN</a></li>
 <li class="toctree-l2"><a class="reference internal" href="compute_server.html#choosing-a-cloud">Choosing a Cloud</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="compute_server.html#cost">Cost</a></li>
 <li class="toctree-l3"><a class="reference internal" href="compute_server.html#availability">Availability</a></li>