git.m455.casa

m455.casa

clone url: git://git.m455.casa/m455.casa

posts/meet-lol-my-new-website-generator.txt

1 title: meet lol, my new website generator!
2
3 2022-08-08 00:00
4
5 i've been pretty busy lately, so i haven't had a lot of time to explore
6 computers, which is what i love doing. in the last few months, i've been trying
7 to fight back against busyness by writing a new website generator after
8 everyone goes to bed. i can't recommend doing this, because it gets exhausting
9 after a while, but it gives me my kicks and makes me happy.
10
11 my old website generator, [https://git.m455.casa/wg|wg], was a wrapper around
12 pandoc, and was written in [https://fennel-lang.org/|fennel]. i used separate
13 fennel scripts to generate a list of posts and an rss feed as post-thoughts to
14 the website generator. also, I didn't know about
15 [https://www.wireguard.com/|wireguard] when I programmed `wg`. for those of you
16 who don't know, wireguard uses the command-line name `wg`, so it was best that
17 i didn't compete with that haha.
18
19 i still love pandoc and fennel, but I wanted to try to program something that
20 had the following features:
21
22 - uses a configuration file
23 - copies the directory structure from the source directory to the build
24 directory
25 - generates a list of posts from a directory that's specified in the
26 configuration file
27 - generates an RSS feed of the posts and their content.
28 - replaces `{{variables}}` in markdown files with values that are declared in
29 the configuration file
30 - provides HTML and RSS templates
31
32 i also wanted an excuse to make a bigger programming project in
33 [https://call-cc.org/|chicken scheme] haha.
34
35 i think what I'm most proud of for this project is that I was able to implement
36 string templates. for example, the `{{im-an-example}}` in the text below would
37 be replaced with the value that corresponds to the `im-an-example` key in a
38 `config.scm` file.
39
40 ```
41 Hey there, this a sentence, and my name is {{im-an-example}}.
42 ```
43
44 though, this wasn't that easy.
45
46 first, i had to implement string replacement... okay, okay, string replacement
47 exists in chicken scheme using the `string-translate`, `string-translate*`,
48 `irregex-replace`, or `irregex-replace/all` procedures, but where's the fun in
49 using those? i don't get to build anything!
50
51 my first step was to write a procedure that replaced the first occurrence of a
52 string. i ended up using the `string-append`, `substring`, and `string-length`
53 procedures to implement the following procedure:
54
55 ```
56 (define (str-replace str from-str to-str)
57 (let* ((from-index (string-contains str from-str))
58 (index-offset (+ from-index (string-length from-str))))
59 (if from-index
60 (string-append (substring str 0 from-index)
61 to-str
62 (substring str
63 index-offset
64 (string-length str)))
65 str)))
66 ```
67
68 which isn't very useful if you plan on having several of the same placeholder
69 values in one string, so i also needed to write a procedure to replace all
70 occurrences of the string. it will drop into an infinite loop if i try to
71 replace `l` with `ll`, but this is personal programming, not some software that
72 needs to be battle tested, so i settled with my implementation below:
73
74 ```
75 (define (str-replace-all str from-str to-str)
76 (let* ((from-index (string-contains str from-str))
77 (index-offset (+ from-index (string-length from-str))))
78 (if from-index
79 (let ((rest-of-string (substring str
80 index-offset
81 (string-length str))))
82 (string-append
83 (substring str 0 from-index)
84 to-str
85 (str-replace-all rest-of-string from-str to-str)))
86 str)))
87 ```
88
89 next, i needed somehow to take a list of pairs, convert the first item in each
90 pair to a string, and then surround the string with `{{` and `}}`, so it
91 resembles one of the placeholder values that i mentioned earlier. after it
92 changed the first element in each pair, i then took the first element of each
93 pair, searched for it in the provided string, and then replaced it with the
94 second element, using the `str-replace-all` procedure to ensure all instances
95 of that placeholder were replaced.
96
97 i actually ended up having to split this algorithm into two procedures to keep
98 things maintainable for myself in case i needed to go back to fix or update the
99 code around this functionality. here are those two procedures:
100
101 ```
102 (define (key->mustached-key pair)
103 (if (pair? pair)
104 (let* ((key (symbol->string (car pair)))
105 (mustached-key (string-append "{{" key "}}"))
106 (value (cadr pair)))
107 `(,mustached-key ,value))
108 pair))
109
110 (define (string-populate str alist)
111 (if (null? alist)
112 str
113 (let* ((mustached-keys (map key->mustached-key alist))
114 (first-pair (car mustached-keys))
115 (key (car first-pair))
116 (val (cadr first-pair)))
117 (string-populate
118 (str-replace-all str key val)
119 (cdr alist)))))
120 ```
121
122 this ended up helping me get really good at quasiquoting in scheme as well!
123
124 apart from the `string-populate` procedure, and the core procedures that it's
125 built on, most of the other features aren't anything special, though i did
126 enjoy that i can just read arbitrary s-expressions from a string using scheme's
127 `read` procedure. the `read` procedure made it super easy to read a
128 configuration file that was all s-expressions. for example, all i needed to do
129 was load an
130 [https://www.gnu.org/software/mit-scheme/documentation/stable/mit-scheme-ref/Association-Lists.html|alist]
131 in a file with the following procedure:
132
133 ```
134 (define (load-config-file)
135 (if (file-exists? config-file)
136 (with-input-from-file config-file read)
137 #f))
138 ```
139
140 this procedure returns a quoted alist, so i wrote the following helper
141 procedure to read it:
142
143 ```
144 (define (get alist key)
145 (if (and (pair? alist)
146 (pair? (car alist))
147 (symbol? key))
148 (cadr (assq key alist))
149 alist))
150 ```
151
152 functional programming purists will hate me for this, but this then allowed me
153 set a globally mutated variable with `(set! config-data (load-config-file))`,
154 and then read the variable with a `(get config-data 'source-dir)`.
155
156 i've been using this method for reading and reloading configuration files for
157 other projects as well, so that was a great learning experience.
158
159 as for generating my list of posts and rss feed, all i needed to do was parse
160 each markdown file in a directory that's specified in the configuration file.
161 to make things easy, the title of a post was extracted from the first line of a
162 file, which should always be a markdown h1 heading. i would then take the
163 markdown heading, for example, `# hey i'm a heading`, and remove the number
164 sign and space proceeding the number sign, leaving me with `hey i'm a heading`.
165
166 the remaining string would be used as the title for each post in the list of
167 posts page, and the title of each rss item. the way i generated links for my
168 list of posts page was by converting the source path from, as an example,
169 `<source-dir>/path/to/post.md` to `https://<domain>/path/to/post.html`.
170
171 because dates are pretty important to rss feeds, although not required, if
172 you're following the spec, i chose to put dates on the third line of each post,
173 in the format of `yyyy-mm-dd`, so i could convert `yyyy-mm-dd` to a number that
174 resembled `yyyymmdd`, and then reverse sort by each number, resulting in a
175 "latest post first, oldest post last" order.
176
177 to kind of finish this off, i think one of the major annoyances was converting
178 all fenced code blocks to use indentation instead, because chicken scheme's
179 lowdown egg replicates what the original markdown parser does. that, and
180 replacing all of my pandoc-centric markdown stuff such as its markdown version
181 of `<div>` blocks:
182
183 ```
184 :::{.im-a-class}
185 hey im a div
186 :::
187 ```
188
189 the upside to using old school, feature-less markdown is that the markdown for
190 my website will work on most markdown parsers i guess? haha.
191
192 the downside to using the lowdown markdown parser is that heading anchors
193 aren't generated, so all of my links to heading anchors are broken, but i got
194 to have fun with programming in scheme at least? plus, this isn't my
195 professional website, so things are allowed to be broken here, and i don't want
196 to get rid of old posts because they bring back good programming adventure
197 memories for me.
198
199 i figured this blog could use a new post, so here it is!
200
201 have a good one!
202
203 if you want to check out the source code for my new website generator, you can
204 view it [https://git.m455.casa/lol/|here].