clone url: git://git.m455.casa/m455.casa
html/archive/2021/setting-up-a-paste-service-with-vim-rsync-and-nginx.html
1 | <!DOCTYPE html> |
2 | <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> |
3 | <head> |
4 | <meta charset="utf-8" /> |
5 | <meta name="generator" content="pandoc" /> |
6 | <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" /> |
7 | <title>Setting up a paste service with Vim, rsync, and nginx</title> |
8 | <style> |
9 | code{white-space: pre-wrap;} |
10 | span.smallcaps{font-variant: small-caps;} |
11 | span.underline{text-decoration: underline;} |
12 | div.column{display: inline-block; vertical-align: top; width: 50%;} |
13 | div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;} |
14 | ul.task-list{list-style: none;} |
15 | </style> |
16 | <style> |
17 | body { |
18 | line-height: 1.5; |
19 | font-family: sans-serif; |
20 | font-size: 18px; |
21 | margin: 20px auto; |
22 | max-width: 630px; |
23 | } |
24 |
|
25 | a { |
26 | color: blue; |
27 | } |
28 |
|
29 | code, pre { |
30 | background-color: #fddee3; |
31 | font-size: 14px; |
32 | } |
33 |
|
34 | pre { |
35 | padding: 25px 25px; |
36 | overflow: auto; |
37 | } |
38 |
|
39 | pre code { |
40 | white-space: pre; |
41 | } |
42 |
|
43 | img { |
44 | max-width: 100%; |
45 | } |
46 |
|
47 | table { |
48 | border-collapse: collapse; |
49 | } |
50 |
|
51 | table caption { |
52 | font-weight: bold; |
53 | margin: 10px 0px; |
54 | text-align: left; |
55 | } |
56 |
|
57 | th, td { |
58 | border: 1px solid #000; |
59 | padding: 4px; |
60 | } |
61 |
|
62 | blockquote { |
63 | border-left: 3px solid #000; |
64 | padding-left: 10px; |
65 | } |
66 |
|
67 | .border { |
68 | border: 1px solid #000; |
69 | margin: 25px 0px; |
70 | padding: 5px 25px; |
71 | } |
72 |
|
73 | @media only screen and (max-width: 700px) { |
74 | body { |
75 | margin: 10px; |
76 | } |
77 | } |
78 |
|
79 | @media (prefers-color-scheme: dark) { |
80 | body { |
81 | background-color: #111; |
82 | color: #eee; |
83 | } |
84 | a { |
85 | color: #009fff; |
86 | } |
87 | code, pre { |
88 | background-color: #111; |
89 | color: #fd6363; |
90 | } |
91 | pre { |
92 | padding: 15px 25px; |
93 | } |
94 | blockquote { |
95 | border-left: 3px solid #666; |
96 | } |
97 | .border, th, td { |
98 | border: 1px solid #666; |
99 | } |
100 | } |
101 | </style> |
102 | </head> |
103 | <body> |
104 | <main> |
105 | <h2 id="setting-up-a-paste-service-with-vim-rsync-and-nginx">Setting up a paste service with Vim, rsync, and nginx</h2> |
106 | <p>2021-01-14 00:00</p> |
107 | <p>This page will guide you through setting up a paste service with Vim, rsync, and nginx. We will be using Vim’s <code>:TOhtml</code> command to use your current Vim <code>colorscheme</code> to provide syntax highlighting in HTML form.</p> |
108 | <h3 id="page-overview">Page overview</h3> |
109 | <!-- vim-markdown-toc GFM --> |
110 | <ul> |
111 | <li><a href="#reasoning-for-this-guide">Reasoning for this guide</a></li> |
112 | <li><a href="#page-conventions">Page conventions</a></li> |
113 | <li><a href="#assumptions">Assumptions</a></li> |
114 | <li><a href="#requirements-on-your-local-machine">Requirements on your local machine</a></li> |
115 | <li><a href="#preparing-your-server">Preparing your server</a> |
116 | <ul> |
117 | <li><a href="#setting-up-dns-records-on-digitalocean">Setting up DNS records on DigitalOcean</a> |
118 | <ul> |
119 | <li><a href="#to-setup-dns-records-on-digitalocean">To setup DNS records on DigitalOcean</a></li> |
120 | </ul></li> |
121 | <li><a href="#creating-a-new-domain-directory">Creating a new domain directory</a> |
122 | <ul> |
123 | <li><a href="#to-create-a-new-domain-directory">To create a new domain directory</a></li> |
124 | </ul></li> |
125 | <li><a href="#configuring-nginx">Configuring nginx</a> |
126 | <ul> |
127 | <li><a href="#to-configure-nginx">To configure nginx</a></li> |
128 | </ul></li> |
129 | <li><a href="#setting-up-your-paste-service-domain-with-certbot">Setting up your paste service domain with certbot</a> |
130 | <ul> |
131 | <li><a href="#to-setup-your-paste-service-domain-with-certbot">To setup your paste service domain with certbot</a></li> |
132 | </ul></li> |
133 | </ul></li> |
134 | <li><a href="#preparing-your-local-machine">Preparing your local machine</a> |
135 | <ul> |
136 | <li><a href="#creating-a-paste-directory">Creating a paste directory</a> |
137 | <ul> |
138 | <li><a href="#to-create-a-paste-directory">To create a paste directory</a></li> |
139 | </ul></li> |
140 | <li><a href="#creating-a-landing-page-for-the-paste-service">Creating a landing page for the paste service</a> |
141 | <ul> |
142 | <li><a href="#to-create-a-landing-page-for-the-paste-service">To create a landing page for the paste service</a></li> |
143 | </ul></li> |
144 | <li><a href="#setting-up-a-generic-file-pasting-script">Setting up a generic file-pasting script</a> |
145 | <ul> |
146 | <li><a href="#to-setup-a-generic-file-pasting-script">To setup a generic file-pasting script</a></li> |
147 | </ul></li> |
148 | <li><a href="#setting-up-a-generic-file-pasting-script-for-plaintext-files">Setting up a generic file-pasting script for plaintext files</a> |
149 | <ul> |
150 | <li><a href="#to-setup-up-a-generic-file-pasting-script-for-plaintext-files">To setup up a generic file-pasting script for plaintext files</a></li> |
151 | </ul></li> |
152 | <li><a href="#setting-up-a-generic-file-synchronizing-script">Setting up a generic file-synchronizing script</a> |
153 | <ul> |
154 | <li><a href="#to-setup-a-generic-file-synchronizing-script">To setup a generic file-synchronizing script</a></li> |
155 | </ul></li> |
156 | <li><a href="#setting-up-vim">Setting up Vim</a> |
157 | <ul> |
158 | <li><a href="#to-setup-vim">To setup Vim</a></li> |
159 | </ul></li> |
160 | </ul></li> |
161 | <li><a href="#using-the-paste-service">Using the paste service</a> |
162 | <ul> |
163 | <li><a href="#uploading-a-code-snippet-from-vim">Uploading a code snippet from Vim</a> |
164 | <ul> |
165 | <li><a href="#to-uploading-a-code-snippet">To uploading a code snippet</a></li> |
166 | </ul></li> |
167 | <li><a href="#uploading-a-generic-file">Uploading a generic file</a> |
168 | <ul> |
169 | <li><a href="#to-upload-a-generic-file">To upload a generic file</a></li> |
170 | </ul></li> |
171 | <li><a href="#uploading-a-generic-file-in-plaintext-format">Uploading a generic file in plaintext format</a> |
172 | <ul> |
173 | <li><a href="#to-upload-a-generic-file-in-plaintext-format">To upload a generic file in plaintext format</a></li> |
174 | </ul></li> |
175 | <li><a href="#removing-a-file">Removing a file</a> |
176 | <ul> |
177 | <li><a href="#to-remove-a-file">To remove a file</a></li> |
178 | </ul></li> |
179 | </ul></li> |
180 | </ul> |
181 | <!-- vim-markdown-toc --> |
182 | <h3 id="reasoning-for-this-guide">Reasoning for this guide</h3> |
183 | <p>I’m writing this guide, not only because it’s a really basic way to show code to friends, but also because I love writing my own little tools and learning about web servers and network infrastructure.</p> |
184 | <h3 id="page-conventions">Page conventions</h3> |
185 | <ul> |
186 | <li><strong>Note</strong>: Signifies additional information</li> |
187 | <li><strong>Tip</strong>: Signifies an alternative procedure for completing a step</li> |
188 | <li><strong>Warning</strong>: Signifies that damage, such as data loss, may occur</li> |
189 | <li><strong>Example</strong>: Shows how a procedure would be performed in a real scenario</li> |
190 | <li><code>Inline code and code blocks</code>: Signify package names, filenames, file contents, or commands</li> |
191 | <li><code>yourdomain.com</code>: Replace this with your domain name.</li> |
192 | </ul> |
193 | <h3 id="assumptions">Assumptions</h3> |
194 | <p>This guide assumes:</p> |
195 | <ul> |
196 | <li>You are using a Ubuntu server on a DigialOcean droplet</li> |
197 | <li>You are using nginx to serve your web content</li> |
198 | <li>You manage your SSL/TLS certificates with certbot</li> |
199 | <li>You have your domain name setup with DigitalOcean’s name servers</li> |
200 | <li>You have your SSH keys setup with your server</li> |
201 | <li>You have root access to your server</li> |
202 | </ul> |
203 | <h3 id="requirements-on-your-local-machine">Requirements on your local machine</h3> |
204 | <ul> |
205 | <li>rsync</li> |
206 | <li>vim</li> |
207 | </ul> |
208 | <h3 id="preparing-your-server">Preparing your server</h3> |
209 | <p>You will need to create a directory for the files you will paste, a landing page in case users decide to navigate to your <code>paste.</code> subdomain, configure nginx to serve the paste directory, and setup SSL/TLS for your domain.</p> |
210 | <p>This section consists of the following topics:</p> |
211 | <ul> |
212 | <li><a href="#setting-up-dns-records-on-digitalocean">Setting up DNS records on DigitalOcean</a></li> |
213 | <li><a href="#creating-a-new-domain-directory">Creating a new domain directory</a></li> |
214 | <li><a href="#creating-a-landing-page-for-the-paste-service">Creating a landing page for the paste service</a></li> |
215 | <li><a href="#configuring-nginx">Configuring nginx</a></li> |
216 | <li><a href="#setting-up-your-paste-service-domain-with-certbot">Setting up your paste service domain with certbot</a></li> |
217 | </ul> |
218 | <h4 id="setting-up-dns-records-on-digitalocean">Setting up DNS records on DigitalOcean</h4> |
219 | <p>Setting up DNS records for <code>paste.yourdomain.com</code> allows DigitalOcean to know about your new subdomain.</p> |
220 | <h5 id="to-setup-dns-records-on-digitalocean">To setup DNS records on DigitalOcean</h5> |
221 | <ol type="1"> |
222 | <li>Add an A record for <code>paste.yourdomain.com</code> to your DigitalOcean droplet</li> |
223 | <li>Add an AAAA record for <code>paste.yourdomain.com</code> to your DigitalOcean droplet</li> |
224 | </ol> |
225 | <aside class="border"> |
226 | <p> |
227 | <strong>Note</strong>: You may need to wait a few minutes for the DNS records to update. |
228 | </p> |
229 | </aside> |
230 | <h4 id="creating-a-new-domain-directory">Creating a new domain directory</h4> |
231 | <p>To share the files you have pasted, you will need to create a directory for your files to reside in.</p> |
232 | <h5 id="to-create-a-new-domain-directory">To create a new domain directory</h5> |
233 | <ol type="1"> |
234 | <li>SSH into your server</li> |
235 | <li>Run <code>sudo mkdir -p /var/www/paste.yourdomain.com</code></li> |
236 | <li>Run <code>sudo chown -R $USER:$USER /var/www/paste.yourdomain.com</code></li> |
237 | <li>Run <code>sudo chmod -R 755 /var/www/paste.yourdomain.com</code></li> |
238 | </ol> |
239 | <h4 id="configuring-nginx">Configuring nginx</h4> |
240 | <p>You will need to setup a subdomain for your domain in nginx. In this guide, I will be using the domain, <code>yourdomain.com</code>, and the subdomain, <code>paste.</code>. The full URL for our pasting service will be <code>paste.yourdomain.com</code>.</p> |
241 | <p>Ensure you replace <code>yourdomain.com</code> with your own domain when following this guide.</p> |
242 | <h5 id="to-configure-nginx">To configure nginx</h5> |
243 | <ol type="1"> |
244 | <li><p>Add the following in <code>/etc/nginx/sites-available/paste.yourdomain.com</code> as root:</p> |
245 | <pre><code> server { |
246 | server_name paste.yourdomain.com; |
247 |
|
248 | root /var/www/paste.yourdomain.com; |
249 | index index.txt; |
250 |
|
251 | location / { |
252 | try_files $uri $uri/ =404; |
253 | } |
254 | }</code></pre></li> |
255 | <li><p>Run the following command:</p> |
256 | <pre><code> sudo ln -s /etc/nginx/sites-available/paste.yourdomain.com /etc/nginx/sites-enabled/paste.yourdomain.com</code></pre></li> |
257 | <li><p>Run <code>sudo systemctl restart nginx</code></p></li> |
258 | </ol> |
259 | <h4 id="setting-up-your-paste-service-domain-with-certbot">Setting up your paste service domain with certbot</h4> |
260 | <p>Setting up SSL/TLS with certbot will stop browsers from telling the user they are on an insecure connection.</p> |
261 | <h5 id="to-setup-your-paste-service-domain-with-certbot">To setup your paste service domain with certbot</h5> |
262 | <ol type="1"> |
263 | <li>Run <code>sudo certbot</code></li> |
264 | <li>Follow the prompts</li> |
265 | <li>Run <code>sudo systemctl restart nginx</code></li> |
266 | </ol> |
267 | <h3 id="preparing-your-local-machine">Preparing your local machine</h3> |
268 | <p>You will need to create a local paste directory, and add a function and command to your <code>~/.vimrc</code> file.</p> |
269 | <p>This section consists of the following topics:</p> |
270 | <ul> |
271 | <li><a href="#creating-a-paste-directory">Creating a paste directory</a></li> |
272 | <li><a href="#creating-a-landing-page-for-the-paste-service">Creating a landing page for the paste service</a></li> |
273 | <li><a href="#setting-up-a-generic-file-pasting-script">Setting up a generic file-pasting script</a></li> |
274 | <li><a href="#setting-up-a-generic-file-pasting-script-for-plaintext-files">Setting up a generic file-pasting script for plaintext files</a></li> |
275 | <li><a href="#setting-up-a-generic-file-synchronizing-script">Setting up a generic file-synchronizing script</a></li> |
276 | <li><a href="#setting-up-vim">Setting up Vim</a></li> |
277 | </ul> |
278 | <h4 id="creating-a-paste-directory">Creating a paste directory</h4> |
279 | <p>The paste directory allows you to keep a collection of files locally. The local paste directory will synchronize with your remote paste directory using rsync.</p> |
280 | <h5 id="to-create-a-paste-directory">To create a paste directory</h5> |
281 | <ol type="1"> |
282 | <li>Run <code>mkdir -p ~/dev/paste</code></li> |
283 | </ol> |
284 | <h4 id="creating-a-landing-page-for-the-paste-service">Creating a landing page for the paste service</h4> |
285 | <p>Users may get curious and wander away from <code>paste.yourdomain.com/some-file.py</code>. One of the places they may go is <code>paste.yourdomain.com</code>, so creating a landing page for this URL will be nicer than getting a 404 error.</p> |
286 | <h5 id="to-create-a-landing-page-for-the-paste-service">To create a landing page for the paste service</h5> |
287 | <ol type="1"> |
288 | <li><p>Run the following command:</p> |
289 | <pre><code> echo "Hey there! It looks like you found my paste service! Unfortunately, there isn't much here!" > ~/dev/paste/index.txt</code></pre></li> |
290 | </ol> |
291 | <aside class="border"> |
292 | <p> |
293 | <strong>Warning</strong>: If you delete the <code>~/dev/paste/index.txt</code> file, you will delete your landing page the next time you synchronize your <code>~/dev/paste</code> directory. |
294 | </p> |
295 | </aside> |
296 | <h4 id="setting-up-a-generic-file-pasting-script">Setting up a generic file-pasting script</h4> |
297 | <p>This script not only supports code snippets with syntax highlight, but it also supports generic file pasting, such as the pasting of images, gifs, videos, etc.</p> |
298 | <h5 id="to-setup-a-generic-file-pasting-script">To setup a generic file-pasting script</h5> |
299 | <ol type="1"> |
300 | <li><p>Add the following to a file called <code>pupload</code>:</p> |
301 | <pre><code> #!/bin/sh |
302 | file="$@" |
303 | domain="https://paste.yourdomain.com/" |
304 |
|
305 | cp $file ~/dev/paste/$file |
306 | cd ~/dev/paste |
307 | rsync -a --delete ./$file user@yourdomain.com:/var/www/paste.yourdomain.com/ |
308 | echo "Paste URL: $domain$file"</code></pre></li> |
309 | <li><p>Run <code>chmod u+x pupload</code></p></li> |
310 | <li><p>Move the <code>pupload</code> to a directory on your <code>$PATH</code></p></li> |
311 | </ol> |
312 | <h4 id="setting-up-a-generic-file-pasting-script-for-plaintext-files">Setting up a generic file-pasting script for plaintext files</h4> |
313 | <p>This script will append a <code>.txt</code> to the end of the filename, so other people can view the file in their browser without downloading the file.</p> |
314 | <h5 id="to-setup-up-a-generic-file-pasting-script-for-plaintext-files">To setup up a generic file-pasting script for plaintext files</h5> |
315 | <ol type="1"> |
316 | <li><p>Add the following to a file called <code>ptxt</code>:</p> |
317 | <pre><code> file="$@" |
318 | filetxt=$file.txt |
319 | domain="https://paste.yourdomain.com/" |
320 |
|
321 | cp $file ~/dev/paste/$filetxt |
322 | cd ~/dev/paste |
323 | rsync -a --delete ./$filetxt user@yourdomain.com:/var/www/paste.yourdomain.com/ |
324 | echo "Paste URL: $domain$filetxt"</code></pre></li> |
325 | <li><p>Run <code>chmod u+x ptxt</code></p></li> |
326 | <li><p>Move the <code>ptxt</code> to a directory on your <code>$PATH</code></p></li> |
327 | </ol> |
328 | <h4 id="setting-up-a-generic-file-synchronizing-script">Setting up a generic file-synchronizing script</h4> |
329 | <p>When removing or modifying files in your <code>~/dev/paste/</code> directory, you will want to synchronize after, so all files on your server update.</p> |
330 | <h5 id="to-setup-a-generic-file-synchronizing-script">To setup a generic file-synchronizing script</h5> |
331 | <ol type="1"> |
332 | <li><p>Add the following to a file called <code>psync</code>:</p> |
333 | <pre><code> #!/bin/sh |
334 | cd ~/dev/paste |
335 | rsync -a --delete ./ user@yourdomain.com:/var/www/paste.yourdomain.com/ |
336 | echo "Files synchronized"</code></pre></li> |
337 | <li><p>Run <code>chmod u+x psync</code></p></li> |
338 | <li><p>Move the <code>psync</code> to a directory on your <code>$PATH</code></p></li> |
339 | </ol> |
340 | <h4 id="setting-up-vim">Setting up Vim</h4> |
341 | <p>Vim commands make uploading and synchronizing your paste directory to your remote server convenient. They remove the need to exit Vim to run shell scripts.</p> |
342 | <h5 id="to-setup-vim">To setup Vim</h5> |
343 | <ol type="1"> |
344 | <li><p>Add the following contents to your <code>~/.vimrc</code>:</p> |
345 | <pre><code> function! PasteSync() |
346 | call system('psync') |
347 | endfunction |
348 |
|
349 | function! PasteUpload() |
350 | TOhtml |
351 | let filename = expand("%:t") |
352 | silent! execute 'write!' '~/dev/paste/' . filename |
353 | call PasteSync() |
354 | q! |
355 | echo 'Paste link: https://paste.yourdomain.com/' . filename |
356 | endfunction |
357 |
|
358 | command PasteSync call PasteSync() |
359 | command PasteUpload call PasteUpload()</code></pre></li> |
360 | </ol> |
361 | <aside class="border"> |
362 | <p> |
363 | <strong>Note</strong>: You will need to replace <code>user</code> in the <code>PasteSync()</code> function with your own SSH username. |
364 | </p> |
365 | </aside> |
366 | <h3 id="using-the-paste-service">Using the paste service</h3> |
367 | <p>You can use the two commands from the previous section to upload and synchronize files to your remote paste directory.</p> |
368 | <p>This section consists of the following topics:</p> |
369 | <ul> |
370 | <li><a href="#uploading-a-code-snippet-from-vim">Uploading a code snippet from Vim</a></li> |
371 | <li><a href="#uploading-a-generic-file">Uploading a generic file</a></li> |
372 | <li><a href="#uploading-a-generic-file-in-plaintext-format">Uploading a generic file in plaintext format</a></li> |
373 | <li><a href="#removing-a-file">Removing a file</a></li> |
374 | </ul> |
375 | <h4 id="uploading-a-code-snippet-from-vim">Uploading a code snippet from Vim</h4> |
376 | <p>The Vim configuration from the previous section allows us to upload Vim buffers as code snippets.</p> |
377 | <h5 id="to-uploading-a-code-snippet">To uploading a code snippet</h5> |
378 | <ol type="1"> |
379 | <li>Open a file in Vim</li> |
380 | <li>Run <code>:PasteUpload</code></li> |
381 | <li>Share the URL generated at the bottom of the screen</li> |
382 | </ol> |
383 | <h4 id="uploading-a-generic-file">Uploading a generic file</h4> |
384 | <p>The <code>pupload</code> script from the <a href="#setting-up-a-generic-file-pasting-script">Setting up a generic file-pasting script</a> section allows you to paste any file to your server.</p> |
385 | <h5 id="to-upload-a-generic-file">To upload a generic file</h5> |
386 | <ol type="1"> |
387 | <li>Run <code>pupload your-file</code></li> |
388 | </ol> |
389 | <h4 id="uploading-a-generic-file-in-plaintext-format">Uploading a generic file in plaintext format</h4> |
390 | <p>Sometimes, you want to view the contents of a file in your browser. Adding “<code>.txt</code>” at the end of a filename will allow your browser to render the contents of the file if it is in a plaintext format.</p> |
391 | <h5 id="to-upload-a-generic-file-in-plaintext-format">To upload a generic file in plaintext format</h5> |
392 | <ol type="1"> |
393 | <li>Run <code>ptxt your-file</code></li> |
394 | </ol> |
395 | <aside class="border"> |
396 | <p> |
397 | <strong>Note</strong>: This will append a <code>.txt</code> to your file. So, if your file is named <code>code.py</code>, the file will be renamed to <code>code.py.txt</code>. When you click the link that is generated, you can view the contents of the file in your browser, instead of being prompted to download the file. |
398 | </p> |
399 | </aside> |
400 | <h4 id="removing-a-file">Removing a file</h4> |
401 | <p>rsync removes the need to SSH into your server to remove a directory by synchronizing a directory’s state using the <code>--delete</code> flag.</p> |
402 | <h5 id="to-remove-a-file">To remove a file</h5> |
403 | <ol type="1"> |
404 | <li><code>cd ~/dev/paste</code></li> |
405 | <li><code>rm file-you-want-to-remove.py</code></li> |
406 | <li>Run <code>psync</code></li> |
407 | </ol> |
408 | </main> |
409 | </body> |
410 | </html> |