Build static websites using any language
1 How to prepare a polyglot project
2 Writing Racket in Markdown
2.1 Design implications
3 Dependency discovery and processing
3.1 Markdown Handling
3.2 Racket Module Handling
3.3 Default File Handling
4 Accessing shared content
5 Responding to change
6 Extending polyglot
polyglot%
7 Project paths
polyglot-project-directory
path-el/  c
project-rel
assets-rel
dist-rel
polyglot-rel
system-temp-rel
8 Publishing to S3
8.1 Assumptions
8.2 Publication steps
9 License and contributions
7.4

Build static websites using any language

Sage Gerard

 (require polyglot) package: polyglot

This module uses unlike-compiler% to generate static websites written in Markdown and any language supported by Racket.

The author’s website uses polyglot and has a page demonstrating use. The README in the source code also doubles an an example page. The demo command builds the README using polyglot.

  $ raco polyglot demo

Use the demo command to verify if polyglot works on your system. If it works, you will see a "dist" folder appear in your working directory containing a "README.html" file. This directory contains a distribution built from assets.

1 How to prepare a polyglot project

  $ raco polyglot start my-website

  $ raco polyglot build my-website

  $ cd my-website

  $ raco polyglot build .

The "my-website" used in the start command creates a new project directory with example content that uses polyglot’s features. The code inside will contain helpful comments.

polyglot commands expect you to specify a path to a project directory to read, whereas start will try to create a directory with the given name.

When you run the build command, polyglot will read "<project>/assets/index.md" and write content to the "<project>/dist" directory, creating it if it does not already exist.

As with the README, once you build the website you will see a dist directory appear. But instead of the directory appearing in your working directory, it will always appear in the project directory.

In this example, polyglot only parses Markdown, wraps the content in an HTML5 document structure, and writes the resulting HTML to "dist/index.html". You will also see other files because polyglot discovers dependencies of your pages and processes them too.

2 Writing Racket in Markdown

Using script elements means that any Markdown parser can parse pages written for polyglot. polyglot only interprets particular elements within Markdown and does not have any parsing rules of its own.

Markdown supports HTML tags inside its content. polyglot will look for script elements of media type "text/racket" and "application/rackdown", in that order. These elements are henceforth named library and application elements, respectively.

  # My page

  

  Lorem ipsum...

  

  <script type="text/racket" id="components">

  #lang racket/base

  

  (provide title link)

  

  (define title "My page")

  (define (link label to) `(a ((href ,to)) ,label))

  </script>

  

  <script type="application/rackdown" id="main">

  #lang racket/base

  

  (provide layout)

  (require "components.rkt")

  

  (define (layout kids)

    `(html (head (title ,title))

           (body . ,kids)))

  

  (write (link "About" "about.md"))

  (write (link "Contact" "contact.md"))

  </script>

  

  Thanks for visiting, blah blah

polyglot writes each script element of only the listed types to a temporary directory. The name of the file will be id.rkt, where id matches the "id" attribute of the script element. If one is not provided, a unique value will be chosen.

In this example polyglot will consume the "text/racket" script (removing it from the output HTML) and write it to "<tmp>/components.rkt". "<tmp>/main.rkt" will follow, and then execute.

The application script uses The Printer in write mode to emit Tagged X-Expressions. The sequence of elements are collected and will replace the script element in the internal txexpr representing the page.

Additionally, the Racket module from an application element can set a layout for the page using (provide layout).

polyglot will continue processing a page until no application elements remain. You can leverage this to generate application elements within an application element. One use case is creating code samples paired with actual output embedded in the page.

(define code '("#lang racket"
               "(write `(h1 \"Hello, meta.\"))"))
 
(write `(pre . ,code))
(write `(script ((id "example") (type "application/rackdown")) ,@code))

2.1 Design implications

3 Dependency discovery and processing

You may have noticed in an earlier example that links can go to other Markdown files.

(write `(a ((href "about.md"))))

polyglot scans all href and src attribute values in a page once that page no longer has any application or library elements.

If any of those values look like they are meant to be paths (including "file:" URLs), they are considered dependencies of your page.

Note that these values that are either absolute paths on your filesystem, or paths relative to your assets directory (See assets-rel). So in the above example, "about.md" corresponds to a complete path like /home/sage/dir-from-command-line/assets/about.md.

3.1 Markdown Handling

polyglot will process any referenced Markdown files just like the one it processed before. All Markdown files will have their application and library elements expand into tagged X-expressions as normal, and all of their dependencies process in turn.

polyglot writes the resulting content as HTML5 to a file in the dist directory with the same name as the original Markdown file, except with an ".html" extension.

3.2 Racket Module Handling

Any referenced ".rkt" files load via (dynamic-require path 'write-dist-file). If you are writing a website on a live build using the raco polyglot develop command, changes to your Racket dependencies will be captured and reloaded using dynamic-rerequire.

The module must provide write-dist-file as an advance/c procedure. That procedure must write to some file in the dist directory and return a complete path to the new file. polyglot will replace the value of the src or href attribute with a path relative to the distribution for you.

This allows you to turn this...

  <link href="compute-stylesheet.rkt" />

...into this...

  <link href="5f9bb103.css" />

...using this:

"compute-stylesheet.rkt"

#lang racket
 
(provide write-dist-file)
(require file/sha1 polyglot)
 
(define (write-dist-file clear compiler)
  (define css "body { font-size: 20px }\n")
  (define port (open-input-string css))
  (define file-name
    (path-replace-extension
      (substring (sha1 port) 0 8)
      #".css"))
 
  (define path (dist-rel file-name))
  (close-input-port port)
  (display-to-file #:mode 'text #:exists 'replace
                   css path)
  path)

The difference between this approach and writing equivalent Racket code in an application element is when the code runs. This code runs after the dependency discovery phase for a Markdown file, but before writing HTML5 to disk.

You can also use this approach to programmatically add! dependencies to the build.

3.3 Default File Handling

Any other files you reference are copied to the dist directory, such that the file name is the first eight characters of the SHA1 hash of the file content. This is strictly for cache busting.

To customize any of the above behavior, see Extending polyglot

4 Accessing shared content

The temporary directory polyglot uses to store modules will contain a symlink to the directory you specify as your project directory. It will always be named "project". Use this link to access resources that are useful for multiple pages.

  <script type="application/rackdown" id="main">

  #lang racket/base

  (require "project/layouts.rkt")

  (provide (rename-out [two-column layout]))

  </script>

If you are using Windows, polyglot will likely not have permission to create links on your system. In that case you can try running polyglot commands as an Administrator, or granting permission to create links specifically.

5 Responding to change

Use the develop command to rebuild your website in response to changes in assets.

  $ raco polyglot develop .

If a Markdown file changes, dependent markdown files will not rebuild. If any other file changes, dependent markdown files will rebuild starting from the dependency discovery stage.

6 Extending polyglot

polyglot offers sufficient flexibility for authoring content within a page, but it handles web asset dependencies in an opinionated way.

class

polyglot% : class?

  superclass: unlike-compiler%

polyglot’s compiler class. If you wish to extend your website to do things like bundle JavaScript, just subclass polyglot% and follow the documentation for unlike-compiler%.
In the terminology of unlike-assets, polyglot% uses complete paths as clear/c names and its clarify method will map any values of href or src attributes to complete paths if they are readable on your system. This is likely enough for most cases, so you’d probably just want to override delegate to recognize new dependencies.
You can even override Markdown processing entirely, but I wouldn’t recommend it.

7 Project paths

polyglot uses several computed paths. You’ll need them to process project files.

This is the primary directory where polyglot will do its work. It may change according to user preferences and will impact the output of all below procedures:

value

path-el/c : 
(and/c (or/c path-for-some-system? path-string?)
       (not/c complete-path?))

procedure

(project-rel path-element ...)  path?

  path-element : path-el/c

procedure

(assets-rel path-element ...)  path?

  path-element : path-el/c

procedure

(dist-rel path-element ...)  path?

  path-element : path-el/c

procedure

(polyglot-rel path-element ...)  path?

  path-element : path-el/c

procedure

(system-temp-rel path-element ...)  path?

  path-element : path-el/c
These procedures behave like build-path, except each returns a path relative to a different directory:

8 Publishing to S3

Use the publish command to upload your website to AWS S3. polyglot’s implementation assumes you have set up a credentials file and will use read-keys/aws-cli to load them.

  $ raco polyglot publish . my-bucket us-east-2

8.1 Assumptions

If you do not agree with all of that, then use the AWS CLI or the aws/s3 library to upload your dist directory.

8.2 Publication steps

  1. Read all keys in the bucket and compare them to the local contents of the dist directory. Remember the elements that are remote but not local.

  2. Uploads the contents of the dist directory to the root of the bucket, overwriting any objects that already exist.

  3. If --delete-diff is set, delete from the bucket the objects marked in Step 1.

Why delete anything? Because if you want to save space, you’ll notice that polyglot will not emit any file unless it was marked as a dependency. If S3 holds a file that polyglot did not emit, it’s either an old version of a file or it was never referenced as a dependency. Your own pages won’t have broken links internally, but changing the name of a Markdown file or removing an existing HTML file will break external links unless you have a system set up to issue HTTP 301 codes. If you want to ensure no broken links, then do not ever use --delete-diff.

9 License and contributions

polyglot uses the MIT license. Here’s the source code.

I always welcome contributions and feedback. If for any reason there is a problem with the name or license, reach out to me and the matter will be resolved immediately.