clone url: git://git.m455.casa/m455.casa
html/posts/setting-up-a-paste-service-with-vim-rsync-and-nginx.html
| 1 | <!DOCTYPE html> |
| 2 | <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> |
| 3 | <head> |
| 4 | <meta charset="utf-8" /> |
| 5 | <meta name="generator" content="pandoc" /> |
| 6 | <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" /> |
| 7 | <title>Setting up a paste service with Vim, rsync, and nginx</title> |
| 8 | <style> |
| 9 | code{white-space: pre-wrap;} |
| 10 | span.smallcaps{font-variant: small-caps;} |
| 11 | span.underline{text-decoration: underline;} |
| 12 | div.column{display: inline-block; vertical-align: top; width: 50%;} |
| 13 | div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;} |
| 14 | ul.task-list{list-style: none;} |
| 15 | </style> |
| 16 | <link rel="stylesheet" href="/assets/archive.css"> |
| 17 | </head> |
| 18 | <body> |
| 19 | <main> |
| 20 | <h2 id="setting-up-a-paste-service-with-vim-rsync-and-nginx">Setting up a paste service with Vim, rsync, and nginx</h2> |
| 21 | <p>2021-01-14 00:00</p> |
| 22 | <p>This page will guide you through setting up a paste service with Vim, rsync, and nginx. We will be using Vim’s <code>:TOhtml</code> command to use your current Vim <code>colorscheme</code> to provide syntax highlighting in HTML form.</p> |
| 23 | <h3 id="page-overview">Page overview</h3> |
| 24 | <!-- vim-markdown-toc GFM --> |
| 25 | <ul> |
| 26 | <li><a href="#reasoning-for-this-guide">Reasoning for this guide</a></li> |
| 27 | <li><a href="#page-conventions">Page conventions</a></li> |
| 28 | <li><a href="#assumptions">Assumptions</a></li> |
| 29 | <li><a href="#requirements-on-your-local-machine">Requirements on your local machine</a></li> |
| 30 | <li><a href="#preparing-your-server">Preparing your server</a> |
| 31 | <ul> |
| 32 | <li><a href="#setting-up-dns-records-on-digitalocean">Setting up DNS records on DigitalOcean</a> |
| 33 | <ul> |
| 34 | <li><a href="#to-setup-dns-records-on-digitalocean">To setup DNS records on DigitalOcean</a></li> |
| 35 | </ul></li> |
| 36 | <li><a href="#creating-a-new-domain-directory">Creating a new domain directory</a> |
| 37 | <ul> |
| 38 | <li><a href="#to-create-a-new-domain-directory">To create a new domain directory</a></li> |
| 39 | </ul></li> |
| 40 | <li><a href="#configuring-nginx">Configuring nginx</a> |
| 41 | <ul> |
| 42 | <li><a href="#to-configure-nginx">To configure nginx</a></li> |
| 43 | </ul></li> |
| 44 | <li><a href="#setting-up-your-paste-service-domain-with-certbot">Setting up your paste service domain with certbot</a> |
| 45 | <ul> |
| 46 | <li><a href="#to-setup-your-paste-service-domain-with-certbot">To setup your paste service domain with certbot</a></li> |
| 47 | </ul></li> |
| 48 | </ul></li> |
| 49 | <li><a href="#preparing-your-local-machine">Preparing your local machine</a> |
| 50 | <ul> |
| 51 | <li><a href="#creating-a-paste-directory">Creating a paste directory</a> |
| 52 | <ul> |
| 53 | <li><a href="#to-create-a-paste-directory">To create a paste directory</a></li> |
| 54 | </ul></li> |
| 55 | <li><a href="#creating-a-landing-page-for-the-paste-service">Creating a landing page for the paste service</a> |
| 56 | <ul> |
| 57 | <li><a href="#to-create-a-landing-page-for-the-paste-service">To create a landing page for the paste service</a></li> |
| 58 | </ul></li> |
| 59 | <li><a href="#setting-up-a-generic-file-pasting-script">Setting up a generic file-pasting script</a> |
| 60 | <ul> |
| 61 | <li><a href="#to-setup-a-generic-file-pasting-script">To setup a generic file-pasting script</a></li> |
| 62 | </ul></li> |
| 63 | <li><a href="#setting-up-a-generic-file-pasting-script-for-plaintext-files">Setting up a generic file-pasting script for plaintext files</a> |
| 64 | <ul> |
| 65 | <li><a href="#to-setup-up-a-generic-file-pasting-script-for-plaintext-files">To setup up a generic file-pasting script for plaintext files</a></li> |
| 66 | </ul></li> |
| 67 | <li><a href="#setting-up-a-generic-file-synchronizing-script">Setting up a generic file-synchronizing script</a> |
| 68 | <ul> |
| 69 | <li><a href="#to-setup-a-generic-file-synchronizing-script">To setup a generic file-synchronizing script</a></li> |
| 70 | </ul></li> |
| 71 | <li><a href="#setting-up-vim">Setting up Vim</a> |
| 72 | <ul> |
| 73 | <li><a href="#to-setup-vim">To setup Vim</a></li> |
| 74 | </ul></li> |
| 75 | </ul></li> |
| 76 | <li><a href="#using-the-paste-service">Using the paste service</a> |
| 77 | <ul> |
| 78 | <li><a href="#uploading-a-code-snippet-from-vim">Uploading a code snippet from Vim</a> |
| 79 | <ul> |
| 80 | <li><a href="#to-uploading-a-code-snippet">To uploading a code snippet</a></li> |
| 81 | </ul></li> |
| 82 | <li><a href="#uploading-a-generic-file">Uploading a generic file</a> |
| 83 | <ul> |
| 84 | <li><a href="#to-upload-a-generic-file">To upload a generic file</a></li> |
| 85 | </ul></li> |
| 86 | <li><a href="#uploading-a-generic-file-in-plaintext-format">Uploading a generic file in plaintext format</a> |
| 87 | <ul> |
| 88 | <li><a href="#to-upload-a-generic-file-in-plaintext-format">To upload a generic file in plaintext format</a></li> |
| 89 | </ul></li> |
| 90 | <li><a href="#removing-a-file">Removing a file</a> |
| 91 | <ul> |
| 92 | <li><a href="#to-remove-a-file">To remove a file</a></li> |
| 93 | </ul></li> |
| 94 | </ul></li> |
| 95 | </ul> |
| 96 | <!-- vim-markdown-toc --> |
| 97 | <h3 id="reasoning-for-this-guide">Reasoning for this guide</h3> |
| 98 | <p>I’m writing this guide, not only because it’s a really basic way to show code to friends, but also because I love writing my own little tools and learning about web servers and network infrastructure.</p> |
| 99 | <h3 id="page-conventions">Page conventions</h3> |
| 100 | <ul> |
| 101 | <li><strong>Note</strong>: Signifies additional information</li> |
| 102 | <li><strong>Tip</strong>: Signifies an alternative procedure for completing a step</li> |
| 103 | <li><strong>Warning</strong>: Signifies that damage, such as data loss, may occur</li> |
| 104 | <li><strong>Example</strong>: Shows how a procedure would be performed in a real scenario</li> |
| 105 | <li><code>Inline code and code blocks</code>: Signify package names, filenames, file contents, or commands</li> |
| 106 | <li><code>yourdomain.com</code>: Replace this with your domain name.</li> |
| 107 | </ul> |
| 108 | <h3 id="assumptions">Assumptions</h3> |
| 109 | <p>This guide assumes:</p> |
| 110 | <ul> |
| 111 | <li>You are using a Ubuntu server on a DigialOcean droplet</li> |
| 112 | <li>You are using nginx to serve your web content</li> |
| 113 | <li>You manage your SSL/TLS certificates with certbot</li> |
| 114 | <li>You have your domain name setup with DigitalOcean’s name servers</li> |
| 115 | <li>You have your SSH keys setup with your server</li> |
| 116 | <li>You have root access to your server</li> |
| 117 | </ul> |
| 118 | <h3 id="requirements-on-your-local-machine">Requirements on your local machine</h3> |
| 119 | <ul> |
| 120 | <li>rsync</li> |
| 121 | <li>vim</li> |
| 122 | </ul> |
| 123 | <h3 id="preparing-your-server">Preparing your server</h3> |
| 124 | <p>You will need to create a directory for the files you will paste, a landing page in case users decide to navigate to your <code>paste.</code> subdomain, configure nginx to serve the paste directory, and setup SSL/TLS for your domain.</p> |
| 125 | <p>This section consists of the following topics:</p> |
| 126 | <ul> |
| 127 | <li><a href="#setting-up-dns-records-on-digitalocean">Setting up DNS records on DigitalOcean</a></li> |
| 128 | <li><a href="#creating-a-new-domain-directory">Creating a new domain directory</a></li> |
| 129 | <li><a href="#creating-a-landing-page-for-the-paste-service">Creating a landing page for the paste service</a></li> |
| 130 | <li><a href="#configuring-nginx">Configuring nginx</a></li> |
| 131 | <li><a href="#setting-up-your-paste-service-domain-with-certbot">Setting up your paste service domain with certbot</a></li> |
| 132 | </ul> |
| 133 | <h4 id="setting-up-dns-records-on-digitalocean">Setting up DNS records on DigitalOcean</h4> |
| 134 | <p>Setting up DNS records for <code>paste.yourdomain.com</code> allows DigitalOcean to know about your new subdomain.</p> |
| 135 | <h5 id="to-setup-dns-records-on-digitalocean">To setup DNS records on DigitalOcean</h5> |
| 136 | <ol type="1"> |
| 137 | <li>Add an A record for <code>paste.yourdomain.com</code> to your DigitalOcean droplet</li> |
| 138 | <li>Add an AAAA record for <code>paste.yourdomain.com</code> to your DigitalOcean droplet</li> |
| 139 | </ol> |
| 140 | <aside class="border"> |
| 141 | <p> |
| 142 | <strong>Note</strong>: You may need to wait a few minutes for the DNS records to update. |
| 143 | </p> |
| 144 | </aside> |
| 145 | <h4 id="creating-a-new-domain-directory">Creating a new domain directory</h4> |
| 146 | <p>To share the files you have pasted, you will need to create a directory for your files to reside in.</p> |
| 147 | <h5 id="to-create-a-new-domain-directory">To create a new domain directory</h5> |
| 148 | <ol type="1"> |
| 149 | <li>SSH into your server</li> |
| 150 | <li>Run <code>sudo mkdir -p /var/www/paste.yourdomain.com</code></li> |
| 151 | <li>Run <code>sudo chown -R $USER:$USER /var/www/paste.yourdomain.com</code></li> |
| 152 | <li>Run <code>sudo chmod -R 755 /var/www/paste.yourdomain.com</code></li> |
| 153 | </ol> |
| 154 | <h4 id="configuring-nginx">Configuring nginx</h4> |
| 155 | <p>You will need to setup a subdomain for your domain in nginx. In this guide, I will be using the domain, <code>yourdomain.com</code>, and the subdomain, <code>paste.</code>. The full URL for our pasting service will be <code>paste.yourdomain.com</code>.</p> |
| 156 | <p>Ensure you replace <code>yourdomain.com</code> with your own domain when following this guide.</p> |
| 157 | <h5 id="to-configure-nginx">To configure nginx</h5> |
| 158 | <ol type="1"> |
| 159 | <li><p>Add the following in <code>/etc/nginx/sites-available/paste.yourdomain.com</code> as root:</p> |
| 160 | <pre><code> server { |
| 161 | server_name paste.yourdomain.com; |
| 162 | |
| 163 | root /var/www/paste.yourdomain.com; |
| 164 | index index.txt; |
| 165 | |
| 166 | location / { |
| 167 | try_files $uri $uri/ =404; |
| 168 | } |
| 169 | }</code></pre></li> |
| 170 | <li><p>Run the following command:</p> |
| 171 | <pre><code> sudo ln -s /etc/nginx/sites-available/paste.yourdomain.com /etc/nginx/sites-enabled/paste.yourdomain.com</code></pre></li> |
| 172 | <li><p>Run <code>sudo systemctl restart nginx</code></p></li> |
| 173 | </ol> |
| 174 | <h4 id="setting-up-your-paste-service-domain-with-certbot">Setting up your paste service domain with certbot</h4> |
| 175 | <p>Setting up SSL/TLS with certbot will stop browsers from telling the user they are on an insecure connection.</p> |
| 176 | <h5 id="to-setup-your-paste-service-domain-with-certbot">To setup your paste service domain with certbot</h5> |
| 177 | <ol type="1"> |
| 178 | <li>Run <code>sudo certbot</code></li> |
| 179 | <li>Follow the prompts</li> |
| 180 | <li>Run <code>sudo systemctl restart nginx</code></li> |
| 181 | </ol> |
| 182 | <h3 id="preparing-your-local-machine">Preparing your local machine</h3> |
| 183 | <p>You will need to create a local paste directory, and add a function and command to your <code>~/.vimrc</code> file.</p> |
| 184 | <p>This section consists of the following topics:</p> |
| 185 | <ul> |
| 186 | <li><a href="#creating-a-paste-directory">Creating a paste directory</a></li> |
| 187 | <li><a href="#creating-a-landing-page-for-the-paste-service">Creating a landing page for the paste service</a></li> |
| 188 | <li><a href="#setting-up-a-generic-file-pasting-script">Setting up a generic file-pasting script</a></li> |
| 189 | <li><a href="#setting-up-a-generic-file-pasting-script-for-plaintext-files">Setting up a generic file-pasting script for plaintext files</a></li> |
| 190 | <li><a href="#setting-up-a-generic-file-synchronizing-script">Setting up a generic file-synchronizing script</a></li> |
| 191 | <li><a href="#setting-up-vim">Setting up Vim</a></li> |
| 192 | </ul> |
| 193 | <h4 id="creating-a-paste-directory">Creating a paste directory</h4> |
| 194 | <p>The paste directory allows you to keep a collection of files locally. The local paste directory will synchronize with your remote paste directory using rsync.</p> |
| 195 | <h5 id="to-create-a-paste-directory">To create a paste directory</h5> |
| 196 | <ol type="1"> |
| 197 | <li>Run <code>mkdir -p ~/dev/paste</code></li> |
| 198 | </ol> |
| 199 | <h4 id="creating-a-landing-page-for-the-paste-service">Creating a landing page for the paste service</h4> |
| 200 | <p>Users may get curious and wander away from <code>paste.yourdomain.com/some-file.py</code>. One of the places they may go is <code>paste.yourdomain.com</code>, so creating a landing page for this URL will be nicer than getting a 404 error.</p> |
| 201 | <h5 id="to-create-a-landing-page-for-the-paste-service">To create a landing page for the paste service</h5> |
| 202 | <ol type="1"> |
| 203 | <li><p>Run the following command:</p> |
| 204 | <pre><code> echo "Hey there! It looks like you found my paste service! Unfortunately, there isn't much here!" > ~/dev/paste/index.txt</code></pre></li> |
| 205 | </ol> |
| 206 | <aside class="border"> |
| 207 | <p> |
| 208 | <strong>Warning</strong>: If you delete the <code>~/dev/paste/index.txt</code> file, you will delete your landing page the next time you synchronize your <code>~/dev/paste</code> directory. |
| 209 | </p> |
| 210 | </aside> |
| 211 | <h4 id="setting-up-a-generic-file-pasting-script">Setting up a generic file-pasting script</h4> |
| 212 | <p>This script not only supports code snippets with syntax highlight, but it also supports generic file pasting, such as the pasting of images, gifs, videos, etc.</p> |
| 213 | <h5 id="to-setup-a-generic-file-pasting-script">To setup a generic file-pasting script</h5> |
| 214 | <ol type="1"> |
| 215 | <li><p>Add the following to a file called <code>pupload</code>:</p> |
| 216 | <pre><code> #!/bin/sh |
| 217 | file="$@" |
| 218 | domain="https://paste.yourdomain.com/" |
| 219 | |
| 220 | cp $file ~/dev/paste/$file |
| 221 | cd ~/dev/paste |
| 222 | rsync -a --delete ./$file user@yourdomain.com:/var/www/paste.yourdomain.com/ |
| 223 | echo "Paste URL: $domain$file"</code></pre></li> |
| 224 | <li><p>Run <code>chmod u+x pupload</code></p></li> |
| 225 | <li><p>Move the <code>pupload</code> to a directory on your <code>$PATH</code></p></li> |
| 226 | </ol> |
| 227 | <h4 id="setting-up-a-generic-file-pasting-script-for-plaintext-files">Setting up a generic file-pasting script for plaintext files</h4> |
| 228 | <p>This script will append a <code>.txt</code> to the end of the filename, so other people can view the file in their browser without downloading the file.</p> |
| 229 | <h5 id="to-setup-up-a-generic-file-pasting-script-for-plaintext-files">To setup up a generic file-pasting script for plaintext files</h5> |
| 230 | <ol type="1"> |
| 231 | <li><p>Add the following to a file called <code>ptxt</code>:</p> |
| 232 | <pre><code> file="$@" |
| 233 | filetxt=$file.txt |
| 234 | domain="https://paste.yourdomain.com/" |
| 235 | |
| 236 | cp $file ~/dev/paste/$filetxt |
| 237 | cd ~/dev/paste |
| 238 | rsync -a --delete ./$filetxt user@yourdomain.com:/var/www/paste.yourdomain.com/ |
| 239 | echo "Paste URL: $domain$filetxt"</code></pre></li> |
| 240 | <li><p>Run <code>chmod u+x ptxt</code></p></li> |
| 241 | <li><p>Move the <code>ptxt</code> to a directory on your <code>$PATH</code></p></li> |
| 242 | </ol> |
| 243 | <h4 id="setting-up-a-generic-file-synchronizing-script">Setting up a generic file-synchronizing script</h4> |
| 244 | <p>When removing or modifying files in your <code>~/dev/paste/</code> directory, you will want to synchronize after, so all files on your server update.</p> |
| 245 | <h5 id="to-setup-a-generic-file-synchronizing-script">To setup a generic file-synchronizing script</h5> |
| 246 | <ol type="1"> |
| 247 | <li><p>Add the following to a file called <code>psync</code>:</p> |
| 248 | <pre><code> #!/bin/sh |
| 249 | cd ~/dev/paste |
| 250 | rsync -a --delete ./ user@yourdomain.com:/var/www/paste.yourdomain.com/ |
| 251 | echo "Files synchronized"</code></pre></li> |
| 252 | <li><p>Run <code>chmod u+x psync</code></p></li> |
| 253 | <li><p>Move the <code>psync</code> to a directory on your <code>$PATH</code></p></li> |
| 254 | </ol> |
| 255 | <h4 id="setting-up-vim">Setting up Vim</h4> |
| 256 | <p>Vim commands make uploading and synchronizing your paste directory to your remote server convenient. They remove the need to exit Vim to run shell scripts.</p> |
| 257 | <h5 id="to-setup-vim">To setup Vim</h5> |
| 258 | <ol type="1"> |
| 259 | <li><p>Add the following contents to your <code>~/.vimrc</code>:</p> |
| 260 | <pre><code> function! PasteSync() |
| 261 | call system('psync') |
| 262 | endfunction |
| 263 | |
| 264 | function! PasteUpload() |
| 265 | TOhtml |
| 266 | let filename = expand("%:t") |
| 267 | silent! execute 'write!' '~/dev/paste/' . filename |
| 268 | call PasteSync() |
| 269 | q! |
| 270 | echo 'Paste link: https://paste.yourdomain.com/' . filename |
| 271 | endfunction |
| 272 | |
| 273 | command PasteSync call PasteSync() |
| 274 | command PasteUpload call PasteUpload()</code></pre></li> |
| 275 | </ol> |
| 276 | <aside class="border"> |
| 277 | <p> |
| 278 | <strong>Note</strong>: You will need to replace <code>user</code> in the <code>PasteSync()</code> function with your own SSH username. |
| 279 | </p> |
| 280 | </aside> |
| 281 | <h3 id="using-the-paste-service">Using the paste service</h3> |
| 282 | <p>You can use the two commands from the previous section to upload and synchronize files to your remote paste directory.</p> |
| 283 | <p>This section consists of the following topics:</p> |
| 284 | <ul> |
| 285 | <li><a href="#uploading-a-code-snippet-from-vim">Uploading a code snippet from Vim</a></li> |
| 286 | <li><a href="#uploading-a-generic-file">Uploading a generic file</a></li> |
| 287 | <li><a href="#uploading-a-generic-file-in-plaintext-format">Uploading a generic file in plaintext format</a></li> |
| 288 | <li><a href="#removing-a-file">Removing a file</a></li> |
| 289 | </ul> |
| 290 | <h4 id="uploading-a-code-snippet-from-vim">Uploading a code snippet from Vim</h4> |
| 291 | <p>The Vim configuration from the previous section allows us to upload Vim buffers as code snippets.</p> |
| 292 | <h5 id="to-uploading-a-code-snippet">To uploading a code snippet</h5> |
| 293 | <ol type="1"> |
| 294 | <li>Open a file in Vim</li> |
| 295 | <li>Run <code>:PasteUpload</code></li> |
| 296 | <li>Share the URL generated at the bottom of the screen</li> |
| 297 | </ol> |
| 298 | <h4 id="uploading-a-generic-file">Uploading a generic file</h4> |
| 299 | <p>The <code>pupload</code> script from the <a href="#setting-up-a-generic-file-pasting-script">Setting up a generic file-pasting script</a> section allows you to paste any file to your server.</p> |
| 300 | <h5 id="to-upload-a-generic-file">To upload a generic file</h5> |
| 301 | <ol type="1"> |
| 302 | <li>Run <code>pupload your-file</code></li> |
| 303 | </ol> |
| 304 | <h4 id="uploading-a-generic-file-in-plaintext-format">Uploading a generic file in plaintext format</h4> |
| 305 | <p>Sometimes, you want to view the contents of a file in your browser. Adding “<code>.txt</code>” at the end of a filename will allow your browser to render the contents of the file if it is in a plaintext format.</p> |
| 306 | <h5 id="to-upload-a-generic-file-in-plaintext-format">To upload a generic file in plaintext format</h5> |
| 307 | <ol type="1"> |
| 308 | <li>Run <code>ptxt your-file</code></li> |
| 309 | </ol> |
| 310 | <aside class="border"> |
| 311 | <p> |
| 312 | <strong>Note</strong>: This will append a <code>.txt</code> to your file. So, if your file is named <code>code.py</code>, the file will be renamed to <code>code.py.txt</code>. When you click the link that is generated, you can view the contents of the file in your browser, instead of being prompted to download the file. |
| 313 | </p> |
| 314 | </aside> |
| 315 | <h4 id="removing-a-file">Removing a file</h4> |
| 316 | <p>rsync removes the need to SSH into your server to remove a directory by synchronizing a directory’s state using the <code>--delete</code> flag.</p> |
| 317 | <h5 id="to-remove-a-file">To remove a file</h5> |
| 318 | <ol type="1"> |
| 319 | <li><code>cd ~/dev/paste</code></li> |
| 320 | <li><code>rm file-you-want-to-remove.py</code></li> |
| 321 | <li>Run <code>psync</code></li> |
| 322 | </ol> |
| 323 | </main> |
| 324 | </body> |
| 325 | </html> |