clone url: git://git.m455.casa/m455.casa
html/posts/setting-up-a-paste-service-with-vim-rsync-and-nginx.html
1 | <!DOCTYPE html> |
2 | <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> |
3 | <head> |
4 | <meta charset="utf-8" /> |
5 | <meta name="generator" content="pandoc" /> |
6 | <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" /> |
7 | <title>Setting up a paste service with Vim, rsync, and nginx</title> |
8 | <style> |
9 | code{white-space: pre-wrap;} |
10 | span.smallcaps{font-variant: small-caps;} |
11 | span.underline{text-decoration: underline;} |
12 | div.column{display: inline-block; vertical-align: top; width: 50%;} |
13 | div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;} |
14 | ul.task-list{list-style: none;} |
15 | </style> |
16 | <link rel="stylesheet" href="/assets/archive.css"> |
17 | </head> |
18 | <body> |
19 | <main> |
20 | <h2 id="setting-up-a-paste-service-with-vim-rsync-and-nginx">Setting up a paste service with Vim, rsync, and nginx</h2> |
21 | <p>2021-01-14 00:00</p> |
22 | <p>This page will guide you through setting up a paste service with Vim, rsync, and nginx. We will be using Vim’s <code>:TOhtml</code> command to use your current Vim <code>colorscheme</code> to provide syntax highlighting in HTML form.</p> |
23 | <h3 id="page-overview">Page overview</h3> |
24 | <!-- vim-markdown-toc GFM --> |
25 | <ul> |
26 | <li><a href="#reasoning-for-this-guide">Reasoning for this guide</a></li> |
27 | <li><a href="#page-conventions">Page conventions</a></li> |
28 | <li><a href="#assumptions">Assumptions</a></li> |
29 | <li><a href="#requirements-on-your-local-machine">Requirements on your local machine</a></li> |
30 | <li><a href="#preparing-your-server">Preparing your server</a> |
31 | <ul> |
32 | <li><a href="#setting-up-dns-records-on-digitalocean">Setting up DNS records on DigitalOcean</a> |
33 | <ul> |
34 | <li><a href="#to-setup-dns-records-on-digitalocean">To setup DNS records on DigitalOcean</a></li> |
35 | </ul></li> |
36 | <li><a href="#creating-a-new-domain-directory">Creating a new domain directory</a> |
37 | <ul> |
38 | <li><a href="#to-create-a-new-domain-directory">To create a new domain directory</a></li> |
39 | </ul></li> |
40 | <li><a href="#configuring-nginx">Configuring nginx</a> |
41 | <ul> |
42 | <li><a href="#to-configure-nginx">To configure nginx</a></li> |
43 | </ul></li> |
44 | <li><a href="#setting-up-your-paste-service-domain-with-certbot">Setting up your paste service domain with certbot</a> |
45 | <ul> |
46 | <li><a href="#to-setup-your-paste-service-domain-with-certbot">To setup your paste service domain with certbot</a></li> |
47 | </ul></li> |
48 | </ul></li> |
49 | <li><a href="#preparing-your-local-machine">Preparing your local machine</a> |
50 | <ul> |
51 | <li><a href="#creating-a-paste-directory">Creating a paste directory</a> |
52 | <ul> |
53 | <li><a href="#to-create-a-paste-directory">To create a paste directory</a></li> |
54 | </ul></li> |
55 | <li><a href="#creating-a-landing-page-for-the-paste-service">Creating a landing page for the paste service</a> |
56 | <ul> |
57 | <li><a href="#to-create-a-landing-page-for-the-paste-service">To create a landing page for the paste service</a></li> |
58 | </ul></li> |
59 | <li><a href="#setting-up-a-generic-file-pasting-script">Setting up a generic file-pasting script</a> |
60 | <ul> |
61 | <li><a href="#to-setup-a-generic-file-pasting-script">To setup a generic file-pasting script</a></li> |
62 | </ul></li> |
63 | <li><a href="#setting-up-a-generic-file-pasting-script-for-plaintext-files">Setting up a generic file-pasting script for plaintext files</a> |
64 | <ul> |
65 | <li><a href="#to-setup-up-a-generic-file-pasting-script-for-plaintext-files">To setup up a generic file-pasting script for plaintext files</a></li> |
66 | </ul></li> |
67 | <li><a href="#setting-up-a-generic-file-synchronizing-script">Setting up a generic file-synchronizing script</a> |
68 | <ul> |
69 | <li><a href="#to-setup-a-generic-file-synchronizing-script">To setup a generic file-synchronizing script</a></li> |
70 | </ul></li> |
71 | <li><a href="#setting-up-vim">Setting up Vim</a> |
72 | <ul> |
73 | <li><a href="#to-setup-vim">To setup Vim</a></li> |
74 | </ul></li> |
75 | </ul></li> |
76 | <li><a href="#using-the-paste-service">Using the paste service</a> |
77 | <ul> |
78 | <li><a href="#uploading-a-code-snippet-from-vim">Uploading a code snippet from Vim</a> |
79 | <ul> |
80 | <li><a href="#to-uploading-a-code-snippet">To uploading a code snippet</a></li> |
81 | </ul></li> |
82 | <li><a href="#uploading-a-generic-file">Uploading a generic file</a> |
83 | <ul> |
84 | <li><a href="#to-upload-a-generic-file">To upload a generic file</a></li> |
85 | </ul></li> |
86 | <li><a href="#uploading-a-generic-file-in-plaintext-format">Uploading a generic file in plaintext format</a> |
87 | <ul> |
88 | <li><a href="#to-upload-a-generic-file-in-plaintext-format">To upload a generic file in plaintext format</a></li> |
89 | </ul></li> |
90 | <li><a href="#removing-a-file">Removing a file</a> |
91 | <ul> |
92 | <li><a href="#to-remove-a-file">To remove a file</a></li> |
93 | </ul></li> |
94 | </ul></li> |
95 | </ul> |
96 | <!-- vim-markdown-toc --> |
97 | <h3 id="reasoning-for-this-guide">Reasoning for this guide</h3> |
98 | <p>I’m writing this guide, not only because it’s a really basic way to show code to friends, but also because I love writing my own little tools and learning about web servers and network infrastructure.</p> |
99 | <h3 id="page-conventions">Page conventions</h3> |
100 | <ul> |
101 | <li><strong>Note</strong>: Signifies additional information</li> |
102 | <li><strong>Tip</strong>: Signifies an alternative procedure for completing a step</li> |
103 | <li><strong>Warning</strong>: Signifies that damage, such as data loss, may occur</li> |
104 | <li><strong>Example</strong>: Shows how a procedure would be performed in a real scenario</li> |
105 | <li><code>Inline code and code blocks</code>: Signify package names, filenames, file contents, or commands</li> |
106 | <li><code>yourdomain.com</code>: Replace this with your domain name.</li> |
107 | </ul> |
108 | <h3 id="assumptions">Assumptions</h3> |
109 | <p>This guide assumes:</p> |
110 | <ul> |
111 | <li>You are using a Ubuntu server on a DigialOcean droplet</li> |
112 | <li>You are using nginx to serve your web content</li> |
113 | <li>You manage your SSL/TLS certificates with certbot</li> |
114 | <li>You have your domain name setup with DigitalOcean’s name servers</li> |
115 | <li>You have your SSH keys setup with your server</li> |
116 | <li>You have root access to your server</li> |
117 | </ul> |
118 | <h3 id="requirements-on-your-local-machine">Requirements on your local machine</h3> |
119 | <ul> |
120 | <li>rsync</li> |
121 | <li>vim</li> |
122 | </ul> |
123 | <h3 id="preparing-your-server">Preparing your server</h3> |
124 | <p>You will need to create a directory for the files you will paste, a landing page in case users decide to navigate to your <code>paste.</code> subdomain, configure nginx to serve the paste directory, and setup SSL/TLS for your domain.</p> |
125 | <p>This section consists of the following topics:</p> |
126 | <ul> |
127 | <li><a href="#setting-up-dns-records-on-digitalocean">Setting up DNS records on DigitalOcean</a></li> |
128 | <li><a href="#creating-a-new-domain-directory">Creating a new domain directory</a></li> |
129 | <li><a href="#creating-a-landing-page-for-the-paste-service">Creating a landing page for the paste service</a></li> |
130 | <li><a href="#configuring-nginx">Configuring nginx</a></li> |
131 | <li><a href="#setting-up-your-paste-service-domain-with-certbot">Setting up your paste service domain with certbot</a></li> |
132 | </ul> |
133 | <h4 id="setting-up-dns-records-on-digitalocean">Setting up DNS records on DigitalOcean</h4> |
134 | <p>Setting up DNS records for <code>paste.yourdomain.com</code> allows DigitalOcean to know about your new subdomain.</p> |
135 | <h5 id="to-setup-dns-records-on-digitalocean">To setup DNS records on DigitalOcean</h5> |
136 | <ol type="1"> |
137 | <li>Add an A record for <code>paste.yourdomain.com</code> to your DigitalOcean droplet</li> |
138 | <li>Add an AAAA record for <code>paste.yourdomain.com</code> to your DigitalOcean droplet</li> |
139 | </ol> |
140 | <aside class="border"> |
141 | <p> |
142 | <strong>Note</strong>: You may need to wait a few minutes for the DNS records to update. |
143 | </p> |
144 | </aside> |
145 | <h4 id="creating-a-new-domain-directory">Creating a new domain directory</h4> |
146 | <p>To share the files you have pasted, you will need to create a directory for your files to reside in.</p> |
147 | <h5 id="to-create-a-new-domain-directory">To create a new domain directory</h5> |
148 | <ol type="1"> |
149 | <li>SSH into your server</li> |
150 | <li>Run <code>sudo mkdir -p /var/www/paste.yourdomain.com</code></li> |
151 | <li>Run <code>sudo chown -R $USER:$USER /var/www/paste.yourdomain.com</code></li> |
152 | <li>Run <code>sudo chmod -R 755 /var/www/paste.yourdomain.com</code></li> |
153 | </ol> |
154 | <h4 id="configuring-nginx">Configuring nginx</h4> |
155 | <p>You will need to setup a subdomain for your domain in nginx. In this guide, I will be using the domain, <code>yourdomain.com</code>, and the subdomain, <code>paste.</code>. The full URL for our pasting service will be <code>paste.yourdomain.com</code>.</p> |
156 | <p>Ensure you replace <code>yourdomain.com</code> with your own domain when following this guide.</p> |
157 | <h5 id="to-configure-nginx">To configure nginx</h5> |
158 | <ol type="1"> |
159 | <li><p>Add the following in <code>/etc/nginx/sites-available/paste.yourdomain.com</code> as root:</p> |
160 | <pre><code> server { |
161 | server_name paste.yourdomain.com; |
162 |
|
163 | root /var/www/paste.yourdomain.com; |
164 | index index.txt; |
165 |
|
166 | location / { |
167 | try_files $uri $uri/ =404; |
168 | } |
169 | }</code></pre></li> |
170 | <li><p>Run the following command:</p> |
171 | <pre><code> sudo ln -s /etc/nginx/sites-available/paste.yourdomain.com /etc/nginx/sites-enabled/paste.yourdomain.com</code></pre></li> |
172 | <li><p>Run <code>sudo systemctl restart nginx</code></p></li> |
173 | </ol> |
174 | <h4 id="setting-up-your-paste-service-domain-with-certbot">Setting up your paste service domain with certbot</h4> |
175 | <p>Setting up SSL/TLS with certbot will stop browsers from telling the user they are on an insecure connection.</p> |
176 | <h5 id="to-setup-your-paste-service-domain-with-certbot">To setup your paste service domain with certbot</h5> |
177 | <ol type="1"> |
178 | <li>Run <code>sudo certbot</code></li> |
179 | <li>Follow the prompts</li> |
180 | <li>Run <code>sudo systemctl restart nginx</code></li> |
181 | </ol> |
182 | <h3 id="preparing-your-local-machine">Preparing your local machine</h3> |
183 | <p>You will need to create a local paste directory, and add a function and command to your <code>~/.vimrc</code> file.</p> |
184 | <p>This section consists of the following topics:</p> |
185 | <ul> |
186 | <li><a href="#creating-a-paste-directory">Creating a paste directory</a></li> |
187 | <li><a href="#creating-a-landing-page-for-the-paste-service">Creating a landing page for the paste service</a></li> |
188 | <li><a href="#setting-up-a-generic-file-pasting-script">Setting up a generic file-pasting script</a></li> |
189 | <li><a href="#setting-up-a-generic-file-pasting-script-for-plaintext-files">Setting up a generic file-pasting script for plaintext files</a></li> |
190 | <li><a href="#setting-up-a-generic-file-synchronizing-script">Setting up a generic file-synchronizing script</a></li> |
191 | <li><a href="#setting-up-vim">Setting up Vim</a></li> |
192 | </ul> |
193 | <h4 id="creating-a-paste-directory">Creating a paste directory</h4> |
194 | <p>The paste directory allows you to keep a collection of files locally. The local paste directory will synchronize with your remote paste directory using rsync.</p> |
195 | <h5 id="to-create-a-paste-directory">To create a paste directory</h5> |
196 | <ol type="1"> |
197 | <li>Run <code>mkdir -p ~/dev/paste</code></li> |
198 | </ol> |
199 | <h4 id="creating-a-landing-page-for-the-paste-service">Creating a landing page for the paste service</h4> |
200 | <p>Users may get curious and wander away from <code>paste.yourdomain.com/some-file.py</code>. One of the places they may go is <code>paste.yourdomain.com</code>, so creating a landing page for this URL will be nicer than getting a 404 error.</p> |
201 | <h5 id="to-create-a-landing-page-for-the-paste-service">To create a landing page for the paste service</h5> |
202 | <ol type="1"> |
203 | <li><p>Run the following command:</p> |
204 | <pre><code> echo "Hey there! It looks like you found my paste service! Unfortunately, there isn't much here!" > ~/dev/paste/index.txt</code></pre></li> |
205 | </ol> |
206 | <aside class="border"> |
207 | <p> |
208 | <strong>Warning</strong>: If you delete the <code>~/dev/paste/index.txt</code> file, you will delete your landing page the next time you synchronize your <code>~/dev/paste</code> directory. |
209 | </p> |
210 | </aside> |
211 | <h4 id="setting-up-a-generic-file-pasting-script">Setting up a generic file-pasting script</h4> |
212 | <p>This script not only supports code snippets with syntax highlight, but it also supports generic file pasting, such as the pasting of images, gifs, videos, etc.</p> |
213 | <h5 id="to-setup-a-generic-file-pasting-script">To setup a generic file-pasting script</h5> |
214 | <ol type="1"> |
215 | <li><p>Add the following to a file called <code>pupload</code>:</p> |
216 | <pre><code> #!/bin/sh |
217 | file="$@" |
218 | domain="https://paste.yourdomain.com/" |
219 |
|
220 | cp $file ~/dev/paste/$file |
221 | cd ~/dev/paste |
222 | rsync -a --delete ./$file user@yourdomain.com:/var/www/paste.yourdomain.com/ |
223 | echo "Paste URL: $domain$file"</code></pre></li> |
224 | <li><p>Run <code>chmod u+x pupload</code></p></li> |
225 | <li><p>Move the <code>pupload</code> to a directory on your <code>$PATH</code></p></li> |
226 | </ol> |
227 | <h4 id="setting-up-a-generic-file-pasting-script-for-plaintext-files">Setting up a generic file-pasting script for plaintext files</h4> |
228 | <p>This script will append a <code>.txt</code> to the end of the filename, so other people can view the file in their browser without downloading the file.</p> |
229 | <h5 id="to-setup-up-a-generic-file-pasting-script-for-plaintext-files">To setup up a generic file-pasting script for plaintext files</h5> |
230 | <ol type="1"> |
231 | <li><p>Add the following to a file called <code>ptxt</code>:</p> |
232 | <pre><code> file="$@" |
233 | filetxt=$file.txt |
234 | domain="https://paste.yourdomain.com/" |
235 |
|
236 | cp $file ~/dev/paste/$filetxt |
237 | cd ~/dev/paste |
238 | rsync -a --delete ./$filetxt user@yourdomain.com:/var/www/paste.yourdomain.com/ |
239 | echo "Paste URL: $domain$filetxt"</code></pre></li> |
240 | <li><p>Run <code>chmod u+x ptxt</code></p></li> |
241 | <li><p>Move the <code>ptxt</code> to a directory on your <code>$PATH</code></p></li> |
242 | </ol> |
243 | <h4 id="setting-up-a-generic-file-synchronizing-script">Setting up a generic file-synchronizing script</h4> |
244 | <p>When removing or modifying files in your <code>~/dev/paste/</code> directory, you will want to synchronize after, so all files on your server update.</p> |
245 | <h5 id="to-setup-a-generic-file-synchronizing-script">To setup a generic file-synchronizing script</h5> |
246 | <ol type="1"> |
247 | <li><p>Add the following to a file called <code>psync</code>:</p> |
248 | <pre><code> #!/bin/sh |
249 | cd ~/dev/paste |
250 | rsync -a --delete ./ user@yourdomain.com:/var/www/paste.yourdomain.com/ |
251 | echo "Files synchronized"</code></pre></li> |
252 | <li><p>Run <code>chmod u+x psync</code></p></li> |
253 | <li><p>Move the <code>psync</code> to a directory on your <code>$PATH</code></p></li> |
254 | </ol> |
255 | <h4 id="setting-up-vim">Setting up Vim</h4> |
256 | <p>Vim commands make uploading and synchronizing your paste directory to your remote server convenient. They remove the need to exit Vim to run shell scripts.</p> |
257 | <h5 id="to-setup-vim">To setup Vim</h5> |
258 | <ol type="1"> |
259 | <li><p>Add the following contents to your <code>~/.vimrc</code>:</p> |
260 | <pre><code> function! PasteSync() |
261 | call system('psync') |
262 | endfunction |
263 |
|
264 | function! PasteUpload() |
265 | TOhtml |
266 | let filename = expand("%:t") |
267 | silent! execute 'write!' '~/dev/paste/' . filename |
268 | call PasteSync() |
269 | q! |
270 | echo 'Paste link: https://paste.yourdomain.com/' . filename |
271 | endfunction |
272 |
|
273 | command PasteSync call PasteSync() |
274 | command PasteUpload call PasteUpload()</code></pre></li> |
275 | </ol> |
276 | <aside class="border"> |
277 | <p> |
278 | <strong>Note</strong>: You will need to replace <code>user</code> in the <code>PasteSync()</code> function with your own SSH username. |
279 | </p> |
280 | </aside> |
281 | <h3 id="using-the-paste-service">Using the paste service</h3> |
282 | <p>You can use the two commands from the previous section to upload and synchronize files to your remote paste directory.</p> |
283 | <p>This section consists of the following topics:</p> |
284 | <ul> |
285 | <li><a href="#uploading-a-code-snippet-from-vim">Uploading a code snippet from Vim</a></li> |
286 | <li><a href="#uploading-a-generic-file">Uploading a generic file</a></li> |
287 | <li><a href="#uploading-a-generic-file-in-plaintext-format">Uploading a generic file in plaintext format</a></li> |
288 | <li><a href="#removing-a-file">Removing a file</a></li> |
289 | </ul> |
290 | <h4 id="uploading-a-code-snippet-from-vim">Uploading a code snippet from Vim</h4> |
291 | <p>The Vim configuration from the previous section allows us to upload Vim buffers as code snippets.</p> |
292 | <h5 id="to-uploading-a-code-snippet">To uploading a code snippet</h5> |
293 | <ol type="1"> |
294 | <li>Open a file in Vim</li> |
295 | <li>Run <code>:PasteUpload</code></li> |
296 | <li>Share the URL generated at the bottom of the screen</li> |
297 | </ol> |
298 | <h4 id="uploading-a-generic-file">Uploading a generic file</h4> |
299 | <p>The <code>pupload</code> script from the <a href="#setting-up-a-generic-file-pasting-script">Setting up a generic file-pasting script</a> section allows you to paste any file to your server.</p> |
300 | <h5 id="to-upload-a-generic-file">To upload a generic file</h5> |
301 | <ol type="1"> |
302 | <li>Run <code>pupload your-file</code></li> |
303 | </ol> |
304 | <h4 id="uploading-a-generic-file-in-plaintext-format">Uploading a generic file in plaintext format</h4> |
305 | <p>Sometimes, you want to view the contents of a file in your browser. Adding “<code>.txt</code>” at the end of a filename will allow your browser to render the contents of the file if it is in a plaintext format.</p> |
306 | <h5 id="to-upload-a-generic-file-in-plaintext-format">To upload a generic file in plaintext format</h5> |
307 | <ol type="1"> |
308 | <li>Run <code>ptxt your-file</code></li> |
309 | </ol> |
310 | <aside class="border"> |
311 | <p> |
312 | <strong>Note</strong>: This will append a <code>.txt</code> to your file. So, if your file is named <code>code.py</code>, the file will be renamed to <code>code.py.txt</code>. When you click the link that is generated, you can view the contents of the file in your browser, instead of being prompted to download the file. |
313 | </p> |
314 | </aside> |
315 | <h4 id="removing-a-file">Removing a file</h4> |
316 | <p>rsync removes the need to SSH into your server to remove a directory by synchronizing a directory’s state using the <code>--delete</code> flag.</p> |
317 | <h5 id="to-remove-a-file">To remove a file</h5> |
318 | <ol type="1"> |
319 | <li><code>cd ~/dev/paste</code></li> |
320 | <li><code>rm file-you-want-to-remove.py</code></li> |
321 | <li>Run <code>psync</code></li> |
322 | </ol> |
323 | </main> |
324 | </body> |
325 | </html> |