Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to jekyll 4.x #46

Closed
0xdevalias opened this issue Feb 6, 2020 · 6 comments · Fixed by #84
Closed

Upgrade to jekyll 4.x #46

0xdevalias opened this issue Feb 6, 2020 · 6 comments · Fixed by #84
Assignees

Comments

@0xdevalias
Copy link
Owner

0xdevalias commented Feb 6, 2020

@0xdevalias
Copy link
Owner Author

0xdevalias commented Jul 21, 2020

Noting old gem version incompatibilities here for reference:

  • You have requested: jekyll ~> 4.1.0 The bundle currently has jekyll locked at 3.8.0. Try running 'bundle update jekyll'
  • jekyll-compose was resolved to 0.8.0, which depends on jekyll (~> 3.0)

This was done after updating a couple of things already:

⇒  bundle outdated
Fetching gem metadata from https://rubygems.org/........
Fetching gem metadata from https://rubygems.org/.
Resolving dependencies...

Outdated gems included in the bundle:
  ..snip..
  * jekyll (newest 4.1.1, installed 3.8.7, requested ~> 3.8.7) in group "default"
  * jekyll-admin (newest 0.10.2, installed 0.8.0) in group "jekyll_plugins"
  * jekyll-feed (newest 0.15.0, installed 0.9.3) in group "jekyll_plugins"
  * jekyll-github-metadata (newest 2.13.0, installed 2.9.4) in group "jekyll_plugins"
  * jekyll-redirect-from (newest 0.16.0, installed 0.13.0) in group "jekyll_plugins"
  * jekyll-sass-converter (newest 2.1.0, installed 1.5.2)
  * jekyll-seo-tag (newest 2.6.1, installed 2.4.0) in group "jekyll_plugins"
  * jekyll-sitemap (newest 1.4.0, installed 1.2.0) in group "jekyll_plugins"
  * jekyll-twitter-plugin (newest 2.1.0, installed 2.0.0) in group "jekyll_plugins"
  * jekyll-webmention_io (newest 3.3.5, installed 2.9.1) in group "jekyll_plugins"
  ..snip..

  • Resolved?
Ruby Sass has reached end-of-life and should no longer be used.

* If you use Sass as a command-line tool, we recommend using Dart Sass, the new
  primary implementation: https://sass-lang.com/install

* If you use Sass as a plug-in for a Ruby web framework, we recommend using the
  sassc gem: https://github.com/sass/sassc-ruby#readme

* For more details, please refer to the Sass blog:
  https://sass-lang.com/blog/posts/7828841

This is a dependency of jekyll-sass-converter which is bundled with jekyll. The latest version appears to correctly update this to use sassc as recommended:

@0xdevalias
Copy link
Owner Author

0xdevalias commented Jul 21, 2020

⇒  bin/serve
Doing `require 'backports'` is deprecated and will not load any backport in the next major release.
Require just the needed backports instead, or 'backports/latest'.

..snip..

      Generating...
   GitHub Metadata: No GitHub API authentication could be found. Some fields may be missing or have incorrect data.

       Jekyll Feed: Generating feed for posts
Markdown processor: "redcarpet" is not a valid Markdown processor.
                    Available processors are: kramdown

  Conversion error: Jekyll::Converters::Markdown encountered an error while converting '_posts/2011-09-16-etc-initd-blog-start.md':
                    Invalid Markdown processor given: redcarpet
             ERROR: YOUR SITE COULD NOT BE BUILT:
                    ------------------------------------
                    Invalid Markdown processor given: redcarpet
                    ------------------------------------------------
      Jekyll 4.1.1   Please append `--trace` to the `serve` command
                     for any additional information or backtrace.
                    ------------------------------------------------
GitHub Metadata: No GitHub API authentication could be found. Some fields may be missing or have incorrect data.

  • Resolved?

Doing require 'backports' is deprecated and will not load any backport in the next major release. Require just the needed backports instead, or 'backports/latest'.

  • backports is required by sinatra-contrib and packable
    • packable is required by nmatrix: packable (~> 1.3, >= 1.3.5), any version of backports
    • sinatra-contrib is required by jekyll-admin: sinatra-contrib (~> 1.4), backports (>= 2.0)

My guess is this deprecated usage will be part of nmatrix since it's so old/seemingly unmaintained.


  • Resolved?

Markdown processor: "redcarpet" is not a valid Markdown processor. Available processors are: kramdown

Support for redcarpet and other markdown processors was dropped. Need to update our usages to kramdown.

Our Jekyll config will likely require changes too:

redcarpet:
  extensions:
    - "tables"
    - "autolink"
    - "strikethrough"
    - "space_after_headers"
    #- "with_toc_data"
    - "fenced_code_blocks"

Kramdown seems to have a number of other issues too, eg:

Not specifically Kramdown related.. but:


  • Resolved?

GitHub Metadata: No GitHub API authentication could be found. Some fields may be missing or have incorrect data.

With my dotfiles layout, I believe we could also add this to ~/.localrc.

I think using a gitignored .env.local file, and also including a .env.example or similar would probably be useful.. if supported.

@0xdevalias
Copy link
Owner Author

0xdevalias commented Jul 21, 2020

  • Resolved?
/Users/devalias/dev/_sites/devalias.net/_plugins/lazy_tweet_embedding.rb:11: warning: calling URI.open via Kernel#open is deprecated, call URI.open directly or use URI#open

It seems as though this may be the original source for this plugin:

It hasn't been updated since 2017, and requires manually copying the plugin source into our project. We might be better off just removing it and manually using a twitter filter around links instead?

@0xdevalias
Copy link
Owner Author

0xdevalias commented Jul 21, 2020

Kramdown seems to have weird quirks that don't really match to GitHub Flavoured Markdown (GFM) as I would have expected it.

An alternative would be to use commonmark (which also appears to be what GitHub pages uses), but there seem to be 2 main versions that appear to use the same underlying commonmarker dependency, and both plugins seem somewhat outdated/unmaintained.

The github/pages-gem claims that it provides gem versions in line with what GitHub pages uses, but according to github/pages-gem#699 it seems to use Jekyll's default parser (Kramdown), rather than the (presumably still accurate, but seemingly unmaintained) CommonMarkGhPages gem.

Though as I note in github/jekyll-commonmark-ghpages#18:

Though looking at the pages-gem readme shows that we can also check the GH pages dependency versions online:

Which shows a usage of jekyll 3.8.7 and jekyll-commonmark-ghpages 0.1.6, which appears to be the latest released version of this project.


commonmarker configuration:

I explore some relevant sounding choices in github/pages-gem#699 (comment), summarised as:

commonmark:
  options: ["FOOTNOTES", "STRIKETHROUGH_DOUBLE_TILDE", "VALIDATE_UTF8", "GITHUB_PRE_LANG", "HARDBREAKS"]
  extensions: ["autolink", "strikethrough", "table", "tasklist", "tagfilter"]

When I don't have UNSAFE enabled, some of my existing pages fail, instead outputting <!-- raw HTML omitted -->.

The main page that I noticed this on was _posts/2017-07-16-bugcrowd-levelup-2017.md, which appears to use syntax like the following:

{% raw %}
<pre>
Some text here.
</pre>
{% endraw %}

Setting the UNSAFE option 'fixes' this.. but probably isn't an ideal choice.

Searching for <!-- raw HTML omitted --> in the generated pages shows up a number of offending pages:

  • _site/author/devalias/rss.xml
  • _site/coaching/index.html
  • _site/devalias/2012/01/10/plesk-adding-programs-to-a-chrooted-ssh/index.html
    • may have been caused by a typo on a triple backtick code fence?
  • _site/devalias/2013/06/17/gists-on-tumblr/index.html
  • _site/devalias/2013/07/13/nmap-sh-saving-precious-seconds/index.html
  • _site/devalias/2013/08/03/rails-lessons-learned-the-hard-way-1-db-migrate/index.html
  • _site/devalias/2013/08/10/erpscan-automator-because-manual-is-meh/index.html
  • _site/devalias/2013/08/29/reversing-powershell-securestring-for-fun-and-profit/index.html
  • _site/devalias/2013/11/21/vfeed-wrapper-helper-scripts-for-speed-and-efficiency/index.html
  • _site/devalias/2014/04/02/hacking-unicoins-for-fun-and-profit/index.html
  • _site/devalias/2014/05/15/java-scala-future-promise-map-headsplode/index.html
  • _site/devalias/2015/04/20/dogedraw-now-with-more-nyaan/index.html
  • _site/devalias/2016/08/24/starting-a-new-web-application-1-an-exploration-of-options/index.html
  • _site/devalias/2016/09/07/graphql-why-you-should-care/index.html
  • _site/devalias/2017/07/16/bugcrowd-levelup-2017/index.html
    • seems to strip the <a> anchor tags for heading links
    • as well as {% raw %}<pre>foo</pre>{% endraw %} usages (which would be better replaced by triple backticks anyway)
  • _site/devalias/2018/05/13/usb-reverse-engineering-down-the-rabbit-hole/index.html
    • seems to strip the <a> anchor tags for heading links
  • _site/devalias/2018/09/15/forming-serverless-clouds-aws-cloudformation-sam-cdk-amplify/index.html
  • _sites/devalias.net/_site/now/index.html
    • strips the <center> tags and {% include goodreads_reading.html %}
    • strips {% include trakt_watching.html %}
  • _site/personal/interests/dance/index.html
  • as well as pretty much every one of the pages in _sites/devalias.net/_site/tag/*/rss.xml

Based on this, it seems like we may have to use UNSAFE. Though given this is a static site I control, that's probably not the worst thing in the world.

It also appears that the tagfilter extension breaks some of our usages in pages/templates/includes, for examples the scripts/tags used in the goodreads/watching embeds in now.md

  • {% include goodreads_reading.html %}
  • {% include trakt_watching.html %}

Again, given I control the source/inputs to this, there probably isn't a lot of real world risk in removing the tagfilter extension as well.


According to github/pages-gem#651, the best recommended way to support Jekyll 4 on GitHub pages currently is to use GitHub actions:

@0xdevalias
Copy link
Owner Author

Fixed up most of the broken gist embeds, but one page still appears to be having issues, seems it may be a bug in jekyll-gist:

@0xdevalias 0xdevalias linked a pull request Jul 22, 2020 that will close this issue
@0xdevalias
Copy link
Owner Author

0xdevalias commented Jul 22, 2020

We have previously used an inlined plugin called postfiles, that appears as though it was copied from:

This is used with syntax such as the following:

 ![who dares enter my domain]({% postfile cat.jpg %})

And appears to be used in the following posts:

  • _posts/2011-12-30-who-dares-enter-my-domain.md
  • _posts/2013-08-16-rockmelt-blog-rockmelt-browser-retirement-july.md
  • _posts/2014-02-13-y-u-no-update-kali.md
  • _site/devalias/2011/11/07/bye-bye-russian-spam/index.html
  • _site/devalias/2013/11/28/thecus-nas-devalias-says-no/index.html

Both of the .html files appear to contain it as the 'prev/next post' excerpt, so isn't really of concern to us here.


There appears to be a maintained (last commit 2020, last release 2019), gem based jekyll plugin that also provides this functionality, though it differs slightly in it's usage:

Instead of using a _postfiles folder, it appears to just use per-post folders under the _posts directory.

Instead of having to explicitly use a liquid filter (eg. ![who dares enter my domain]({% postfile cat.jpg %})), we can just use a regular markdown image tag with a relative path (eg. ![who dares enter my domain](cat.jpg))

It's probably a better choice to use going forward.

  • Resolved?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant