git.m455.casa

m455.casa

clone url: git://git.m455.casa/m455.casa


html/archive/2021/setting-up-a-paste-service-with-vim-rsync-and-nginx.html

1 <!DOCTYPE html>
2 <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
3 <head>
4 <meta charset="utf-8" />
5 <meta name="generator" content="pandoc" />
6 <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
7 <title>Setting up a paste service with Vim, rsync, and nginx</title>
8 <style>
9 code{white-space: pre-wrap;}
10 span.smallcaps{font-variant: small-caps;}
11 span.underline{text-decoration: underline;}
12 div.column{display: inline-block; vertical-align: top; width: 50%;}
13 div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
14 ul.task-list{list-style: none;}
15 </style>
16 <style>
17 body {
18 line-height: 1.5;
19 font-family: sans-serif;
20 font-size: 18px;
21 margin: 20px auto;
22 max-width: 630px;
23 }
24
25 a {
26 color: blue;
27 }
28
29 code, pre {
30 background-color: #fddee3;
31 font-size: 14px;
32 }
33
34 pre {
35 padding: 25px 25px;
36 overflow: auto;
37 }
38
39 pre code {
40 white-space: pre;
41 }
42
43 img {
44 max-width: 100%;
45 }
46
47 table {
48 border-collapse: collapse;
49 }
50
51 table caption {
52 font-weight: bold;
53 margin: 10px 0px;
54 text-align: left;
55 }
56
57 th, td {
58 border: 1px solid #000;
59 padding: 4px;
60 }
61
62 blockquote {
63 border-left: 3px solid #000;
64 padding-left: 10px;
65 }
66
67 .border {
68 border: 1px solid #000;
69 margin: 25px 0px;
70 padding: 5px 25px;
71 }
72
73 @media only screen and (max-width: 700px) {
74 body {
75 margin: 10px;
76 }
77 }
78
79 @media (prefers-color-scheme: dark) {
80 body {
81 background-color: #111;
82 color: #eee;
83 }
84 a {
85 color: #009fff;
86 }
87 code, pre {
88 background-color: #111;
89 color: #fd6363;
90 }
91 pre {
92 padding: 15px 25px;
93 }
94 blockquote {
95 border-left: 3px solid #666;
96 }
97 .border, th, td {
98 border: 1px solid #666;
99 }
100 }
101 </style>
102 </head>
103 <body>
104 <main>
105 <h2 id="setting-up-a-paste-service-with-vim-rsync-and-nginx">Setting up a paste service with Vim, rsync, and nginx</h2>
106 <p>2021-01-14 00:00</p>
107 <p>This page will guide you through setting up a paste service with Vim, rsync, and nginx. We will be using Vim’s <code>:TOhtml</code> command to use your current Vim <code>colorscheme</code> to provide syntax highlighting in HTML form.</p>
108 <h3 id="page-overview">Page overview</h3>
109 <!-- vim-markdown-toc GFM -->
110 <ul>
111 <li><a href="#reasoning-for-this-guide">Reasoning for this guide</a></li>
112 <li><a href="#page-conventions">Page conventions</a></li>
113 <li><a href="#assumptions">Assumptions</a></li>
114 <li><a href="#requirements-on-your-local-machine">Requirements on your local machine</a></li>
115 <li><a href="#preparing-your-server">Preparing your server</a>
116 <ul>
117 <li><a href="#setting-up-dns-records-on-digitalocean">Setting up DNS records on DigitalOcean</a>
118 <ul>
119 <li><a href="#to-setup-dns-records-on-digitalocean">To setup DNS records on DigitalOcean</a></li>
120 </ul></li>
121 <li><a href="#creating-a-new-domain-directory">Creating a new domain directory</a>
122 <ul>
123 <li><a href="#to-create-a-new-domain-directory">To create a new domain directory</a></li>
124 </ul></li>
125 <li><a href="#configuring-nginx">Configuring nginx</a>
126 <ul>
127 <li><a href="#to-configure-nginx">To configure nginx</a></li>
128 </ul></li>
129 <li><a href="#setting-up-your-paste-service-domain-with-certbot">Setting up your paste service domain with certbot</a>
130 <ul>
131 <li><a href="#to-setup-your-paste-service-domain-with-certbot">To setup your paste service domain with certbot</a></li>
132 </ul></li>
133 </ul></li>
134 <li><a href="#preparing-your-local-machine">Preparing your local machine</a>
135 <ul>
136 <li><a href="#creating-a-paste-directory">Creating a paste directory</a>
137 <ul>
138 <li><a href="#to-create-a-paste-directory">To create a paste directory</a></li>
139 </ul></li>
140 <li><a href="#creating-a-landing-page-for-the-paste-service">Creating a landing page for the paste service</a>
141 <ul>
142 <li><a href="#to-create-a-landing-page-for-the-paste-service">To create a landing page for the paste service</a></li>
143 </ul></li>
144 <li><a href="#setting-up-a-generic-file-pasting-script">Setting up a generic file-pasting script</a>
145 <ul>
146 <li><a href="#to-setup-a-generic-file-pasting-script">To setup a generic file-pasting script</a></li>
147 </ul></li>
148 <li><a href="#setting-up-a-generic-file-pasting-script-for-plaintext-files">Setting up a generic file-pasting script for plaintext files</a>
149 <ul>
150 <li><a href="#to-setup-up-a-generic-file-pasting-script-for-plaintext-files">To setup up a generic file-pasting script for plaintext files</a></li>
151 </ul></li>
152 <li><a href="#setting-up-a-generic-file-synchronizing-script">Setting up a generic file-synchronizing script</a>
153 <ul>
154 <li><a href="#to-setup-a-generic-file-synchronizing-script">To setup a generic file-synchronizing script</a></li>
155 </ul></li>
156 <li><a href="#setting-up-vim">Setting up Vim</a>
157 <ul>
158 <li><a href="#to-setup-vim">To setup Vim</a></li>
159 </ul></li>
160 </ul></li>
161 <li><a href="#using-the-paste-service">Using the paste service</a>
162 <ul>
163 <li><a href="#uploading-a-code-snippet-from-vim">Uploading a code snippet from Vim</a>
164 <ul>
165 <li><a href="#to-uploading-a-code-snippet">To uploading a code snippet</a></li>
166 </ul></li>
167 <li><a href="#uploading-a-generic-file">Uploading a generic file</a>
168 <ul>
169 <li><a href="#to-upload-a-generic-file">To upload a generic file</a></li>
170 </ul></li>
171 <li><a href="#uploading-a-generic-file-in-plaintext-format">Uploading a generic file in plaintext format</a>
172 <ul>
173 <li><a href="#to-upload-a-generic-file-in-plaintext-format">To upload a generic file in plaintext format</a></li>
174 </ul></li>
175 <li><a href="#removing-a-file">Removing a file</a>
176 <ul>
177 <li><a href="#to-remove-a-file">To remove a file</a></li>
178 </ul></li>
179 </ul></li>
180 </ul>
181 <!-- vim-markdown-toc -->
182 <h3 id="reasoning-for-this-guide">Reasoning for this guide</h3>
183 <p>I’m writing this guide, not only because it’s a really basic way to show code to friends, but also because I love writing my own little tools and learning about web servers and network infrastructure.</p>
184 <h3 id="page-conventions">Page conventions</h3>
185 <ul>
186 <li><strong>Note</strong>: Signifies additional information</li>
187 <li><strong>Tip</strong>: Signifies an alternative procedure for completing a step</li>
188 <li><strong>Warning</strong>: Signifies that damage, such as data loss, may occur</li>
189 <li><strong>Example</strong>: Shows how a procedure would be performed in a real scenario</li>
190 <li><code>Inline code and code blocks</code>: Signify package names, filenames, file contents, or commands</li>
191 <li><code>yourdomain.com</code>: Replace this with your domain name.</li>
192 </ul>
193 <h3 id="assumptions">Assumptions</h3>
194 <p>This guide assumes:</p>
195 <ul>
196 <li>You are using a Ubuntu server on a DigialOcean droplet</li>
197 <li>You are using nginx to serve your web content</li>
198 <li>You manage your SSL/TLS certificates with certbot</li>
199 <li>You have your domain name setup with DigitalOcean’s name servers</li>
200 <li>You have your SSH keys setup with your server</li>
201 <li>You have root access to your server</li>
202 </ul>
203 <h3 id="requirements-on-your-local-machine">Requirements on your local machine</h3>
204 <ul>
205 <li>rsync</li>
206 <li>vim</li>
207 </ul>
208 <h3 id="preparing-your-server">Preparing your server</h3>
209 <p>You will need to create a directory for the files you will paste, a landing page in case users decide to navigate to your <code>paste.</code> subdomain, configure nginx to serve the paste directory, and setup SSL/TLS for your domain.</p>
210 <p>This section consists of the following topics:</p>
211 <ul>
212 <li><a href="#setting-up-dns-records-on-digitalocean">Setting up DNS records on DigitalOcean</a></li>
213 <li><a href="#creating-a-new-domain-directory">Creating a new domain directory</a></li>
214 <li><a href="#creating-a-landing-page-for-the-paste-service">Creating a landing page for the paste service</a></li>
215 <li><a href="#configuring-nginx">Configuring nginx</a></li>
216 <li><a href="#setting-up-your-paste-service-domain-with-certbot">Setting up your paste service domain with certbot</a></li>
217 </ul>
218 <h4 id="setting-up-dns-records-on-digitalocean">Setting up DNS records on DigitalOcean</h4>
219 <p>Setting up DNS records for <code>paste.yourdomain.com</code> allows DigitalOcean to know about your new subdomain.</p>
220 <h5 id="to-setup-dns-records-on-digitalocean">To setup DNS records on DigitalOcean</h5>
221 <ol type="1">
222 <li>Add an A record for <code>paste.yourdomain.com</code> to your DigitalOcean droplet</li>
223 <li>Add an AAAA record for <code>paste.yourdomain.com</code> to your DigitalOcean droplet</li>
224 </ol>
225 <aside class="border">
226 <p>
227 <strong>Note</strong>: You may need to wait a few minutes for the DNS records to update.
228 </p>
229 </aside>
230 <h4 id="creating-a-new-domain-directory">Creating a new domain directory</h4>
231 <p>To share the files you have pasted, you will need to create a directory for your files to reside in.</p>
232 <h5 id="to-create-a-new-domain-directory">To create a new domain directory</h5>
233 <ol type="1">
234 <li>SSH into your server</li>
235 <li>Run <code>sudo mkdir -p /var/www/paste.yourdomain.com</code></li>
236 <li>Run <code>sudo chown -R $USER:$USER /var/www/paste.yourdomain.com</code></li>
237 <li>Run <code>sudo chmod -R 755 /var/www/paste.yourdomain.com</code></li>
238 </ol>
239 <h4 id="configuring-nginx">Configuring nginx</h4>
240 <p>You will need to setup a subdomain for your domain in nginx. In this guide, I will be using the domain, <code>yourdomain.com</code>, and the subdomain, <code>paste.</code>. The full URL for our pasting service will be <code>paste.yourdomain.com</code>.</p>
241 <p>Ensure you replace <code>yourdomain.com</code> with your own domain when following this guide.</p>
242 <h5 id="to-configure-nginx">To configure nginx</h5>
243 <ol type="1">
244 <li><p>Add the following in <code>/etc/nginx/sites-available/paste.yourdomain.com</code> as root:</p>
245 <pre><code> server {
246 server_name paste.yourdomain.com;
247
248 root /var/www/paste.yourdomain.com;
249 index index.txt;
250
251 location / {
252 try_files $uri $uri/ =404;
253 }
254 }</code></pre></li>
255 <li><p>Run the following command:</p>
256 <pre><code> sudo ln -s /etc/nginx/sites-available/paste.yourdomain.com /etc/nginx/sites-enabled/paste.yourdomain.com</code></pre></li>
257 <li><p>Run <code>sudo systemctl restart nginx</code></p></li>
258 </ol>
259 <h4 id="setting-up-your-paste-service-domain-with-certbot">Setting up your paste service domain with certbot</h4>
260 <p>Setting up SSL/TLS with certbot will stop browsers from telling the user they are on an insecure connection.</p>
261 <h5 id="to-setup-your-paste-service-domain-with-certbot">To setup your paste service domain with certbot</h5>
262 <ol type="1">
263 <li>Run <code>sudo certbot</code></li>
264 <li>Follow the prompts</li>
265 <li>Run <code>sudo systemctl restart nginx</code></li>
266 </ol>
267 <h3 id="preparing-your-local-machine">Preparing your local machine</h3>
268 <p>You will need to create a local paste directory, and add a function and command to your <code>~/.vimrc</code> file.</p>
269 <p>This section consists of the following topics:</p>
270 <ul>
271 <li><a href="#creating-a-paste-directory">Creating a paste directory</a></li>
272 <li><a href="#creating-a-landing-page-for-the-paste-service">Creating a landing page for the paste service</a></li>
273 <li><a href="#setting-up-a-generic-file-pasting-script">Setting up a generic file-pasting script</a></li>
274 <li><a href="#setting-up-a-generic-file-pasting-script-for-plaintext-files">Setting up a generic file-pasting script for plaintext files</a></li>
275 <li><a href="#setting-up-a-generic-file-synchronizing-script">Setting up a generic file-synchronizing script</a></li>
276 <li><a href="#setting-up-vim">Setting up Vim</a></li>
277 </ul>
278 <h4 id="creating-a-paste-directory">Creating a paste directory</h4>
279 <p>The paste directory allows you to keep a collection of files locally. The local paste directory will synchronize with your remote paste directory using rsync.</p>
280 <h5 id="to-create-a-paste-directory">To create a paste directory</h5>
281 <ol type="1">
282 <li>Run <code>mkdir -p ~/dev/paste</code></li>
283 </ol>
284 <h4 id="creating-a-landing-page-for-the-paste-service">Creating a landing page for the paste service</h4>
285 <p>Users may get curious and wander away from <code>paste.yourdomain.com/some-file.py</code>. One of the places they may go is <code>paste.yourdomain.com</code>, so creating a landing page for this URL will be nicer than getting a 404 error.</p>
286 <h5 id="to-create-a-landing-page-for-the-paste-service">To create a landing page for the paste service</h5>
287 <ol type="1">
288 <li><p>Run the following command:</p>
289 <pre><code> echo &quot;Hey there! It looks like you found my paste service! Unfortunately, there isn&#39;t much here!&quot; &gt; ~/dev/paste/index.txt</code></pre></li>
290 </ol>
291 <aside class="border">
292 <p>
293 <strong>Warning</strong>: If you delete the <code>~/dev/paste/index.txt</code> file, you will delete your landing page the next time you synchronize your <code>~/dev/paste</code> directory.
294 </p>
295 </aside>
296 <h4 id="setting-up-a-generic-file-pasting-script">Setting up a generic file-pasting script</h4>
297 <p>This script not only supports code snippets with syntax highlight, but it also supports generic file pasting, such as the pasting of images, gifs, videos, etc.</p>
298 <h5 id="to-setup-a-generic-file-pasting-script">To setup a generic file-pasting script</h5>
299 <ol type="1">
300 <li><p>Add the following to a file called <code>pupload</code>:</p>
301 <pre><code> #!/bin/sh
302 file=&quot;$@&quot;
303 domain=&quot;https://paste.yourdomain.com/&quot;
304
305 cp $file ~/dev/paste/$file
306 cd ~/dev/paste
307 rsync -a --delete ./$file user@yourdomain.com:/var/www/paste.yourdomain.com/
308 echo &quot;Paste URL: $domain$file&quot;</code></pre></li>
309 <li><p>Run <code>chmod u+x pupload</code></p></li>
310 <li><p>Move the <code>pupload</code> to a directory on your <code>$PATH</code></p></li>
311 </ol>
312 <h4 id="setting-up-a-generic-file-pasting-script-for-plaintext-files">Setting up a generic file-pasting script for plaintext files</h4>
313 <p>This script will append a <code>.txt</code> to the end of the filename, so other people can view the file in their browser without downloading the file.</p>
314 <h5 id="to-setup-up-a-generic-file-pasting-script-for-plaintext-files">To setup up a generic file-pasting script for plaintext files</h5>
315 <ol type="1">
316 <li><p>Add the following to a file called <code>ptxt</code>:</p>
317 <pre><code> file=&quot;$@&quot;
318 filetxt=$file.txt
319 domain=&quot;https://paste.yourdomain.com/&quot;
320
321 cp $file ~/dev/paste/$filetxt
322 cd ~/dev/paste
323 rsync -a --delete ./$filetxt user@yourdomain.com:/var/www/paste.yourdomain.com/
324 echo &quot;Paste URL: $domain$filetxt&quot;</code></pre></li>
325 <li><p>Run <code>chmod u+x ptxt</code></p></li>
326 <li><p>Move the <code>ptxt</code> to a directory on your <code>$PATH</code></p></li>
327 </ol>
328 <h4 id="setting-up-a-generic-file-synchronizing-script">Setting up a generic file-synchronizing script</h4>
329 <p>When removing or modifying files in your <code>~/dev/paste/</code> directory, you will want to synchronize after, so all files on your server update.</p>
330 <h5 id="to-setup-a-generic-file-synchronizing-script">To setup a generic file-synchronizing script</h5>
331 <ol type="1">
332 <li><p>Add the following to a file called <code>psync</code>:</p>
333 <pre><code> #!/bin/sh
334 cd ~/dev/paste
335 rsync -a --delete ./ user@yourdomain.com:/var/www/paste.yourdomain.com/
336 echo &quot;Files synchronized&quot;</code></pre></li>
337 <li><p>Run <code>chmod u+x psync</code></p></li>
338 <li><p>Move the <code>psync</code> to a directory on your <code>$PATH</code></p></li>
339 </ol>
340 <h4 id="setting-up-vim">Setting up Vim</h4>
341 <p>Vim commands make uploading and synchronizing your paste directory to your remote server convenient. They remove the need to exit Vim to run shell scripts.</p>
342 <h5 id="to-setup-vim">To setup Vim</h5>
343 <ol type="1">
344 <li><p>Add the following contents to your <code>~/.vimrc</code>:</p>
345 <pre><code> function! PasteSync()
346 call system(&#39;psync&#39;)
347 endfunction
348
349 function! PasteUpload()
350 TOhtml
351 let filename = expand(&quot;%:t&quot;)
352 silent! execute &#39;write!&#39; &#39;~/dev/paste/&#39; . filename
353 call PasteSync()
354 q!
355 echo &#39;Paste link: https://paste.yourdomain.com/&#39; . filename
356 endfunction
357
358 command PasteSync call PasteSync()
359 command PasteUpload call PasteUpload()</code></pre></li>
360 </ol>
361 <aside class="border">
362 <p>
363 <strong>Note</strong>: You will need to replace <code>user</code> in the <code>PasteSync()</code> function with your own SSH username.
364 </p>
365 </aside>
366 <h3 id="using-the-paste-service">Using the paste service</h3>
367 <p>You can use the two commands from the previous section to upload and synchronize files to your remote paste directory.</p>
368 <p>This section consists of the following topics:</p>
369 <ul>
370 <li><a href="#uploading-a-code-snippet-from-vim">Uploading a code snippet from Vim</a></li>
371 <li><a href="#uploading-a-generic-file">Uploading a generic file</a></li>
372 <li><a href="#uploading-a-generic-file-in-plaintext-format">Uploading a generic file in plaintext format</a></li>
373 <li><a href="#removing-a-file">Removing a file</a></li>
374 </ul>
375 <h4 id="uploading-a-code-snippet-from-vim">Uploading a code snippet from Vim</h4>
376 <p>The Vim configuration from the previous section allows us to upload Vim buffers as code snippets.</p>
377 <h5 id="to-uploading-a-code-snippet">To uploading a code snippet</h5>
378 <ol type="1">
379 <li>Open a file in Vim</li>
380 <li>Run <code>:PasteUpload</code></li>
381 <li>Share the URL generated at the bottom of the screen</li>
382 </ol>
383 <h4 id="uploading-a-generic-file">Uploading a generic file</h4>
384 <p>The <code>pupload</code> script from the <a href="#setting-up-a-generic-file-pasting-script">Setting up a generic file-pasting script</a> section allows you to paste any file to your server.</p>
385 <h5 id="to-upload-a-generic-file">To upload a generic file</h5>
386 <ol type="1">
387 <li>Run <code>pupload your-file</code></li>
388 </ol>
389 <h4 id="uploading-a-generic-file-in-plaintext-format">Uploading a generic file in plaintext format</h4>
390 <p>Sometimes, you want to view the contents of a file in your browser. Adding “<code>.txt</code>” at the end of a filename will allow your browser to render the contents of the file if it is in a plaintext format.</p>
391 <h5 id="to-upload-a-generic-file-in-plaintext-format">To upload a generic file in plaintext format</h5>
392 <ol type="1">
393 <li>Run <code>ptxt your-file</code></li>
394 </ol>
395 <aside class="border">
396 <p>
397 <strong>Note</strong>: This will append a <code>.txt</code> to your file. So, if your file is named <code>code.py</code>, the file will be renamed to <code>code.py.txt</code>. When you click the link that is generated, you can view the contents of the file in your browser, instead of being prompted to download the file.
398 </p>
399 </aside>
400 <h4 id="removing-a-file">Removing a file</h4>
401 <p>rsync removes the need to SSH into your server to remove a directory by synchronizing a directory’s state using the <code>--delete</code> flag.</p>
402 <h5 id="to-remove-a-file">To remove a file</h5>
403 <ol type="1">
404 <li><code>cd ~/dev/paste</code></li>
405 <li><code>rm file-you-want-to-remove.py</code></li>
406 <li>Run <code>psync</code></li>
407 </ol>
408 </main>
409 </body>
410 </html>