I Built an npm Package to Scrape and Slim Down YAML Files

This article is a translated version of my original post on Qiita. Original (Japanese): https://qiita.com/segur/items/eee22ec02545e2ad3f49

I Built a Library That Scrapes and Slims Down YAML Files

I created an npm module that strips YAML files down to size. It can delete specific elements or replace them with empty values.

Published here:

The word "scraper" means something like "to scrape off." It even has a Travis CI badge!

What Can It Do?

Given a YAML file like this:

sample:
  hoge: I want to delete this
  fuga: I want to empty this
  hogehoge:
    hoge: I want to delete this too
    fuga: I want to empty this too
    hogera: Leave this as-is
  piyopiyo:
    piyo: I want to delete the parent of this

The library can:

Delete elements whose key is hoge
Keep elements whose key is fuga, but replace the value with an empty string
Delete the parent element of the element whose key is piyo

The resulting output is:

sample:
  fuga: ''
  hogehoge:
    fuga: ''
    hogera: Leave this as-is

Unnecessary elements are stripped away, making the YAML file much lighter. It works at any nesting depth!

Why I Built It

I use Swagger to generate API documentation sites. When I tried to import a Swagger file into Amazon API Gateway, I got a size limit error.

To work around that, I built this library to strip out unnecessary elements from the YAML.

Installation

Node.js is required.

If Node.js is installed, run:

npm install yaml-scraper

Sample Code

Here's the sample code for the operations described above:

// Load libraries
const fs = require('fs');
const scraper = require('yaml-scraper');

// Read the YAML file
const input = fs.readFileSync('./sample.yaml', 'utf8');

// Delete 'hoge', empty 'fuga', delete parent of 'piyo'
const output = scraper(input)
  .delete('hoge')
  .empty('fuga')
  .deleteParent('piyo')
  .toString();

// Print the result
console.log(output);

As you can see, delete, empty, and deleteParent are provided as a method chain.

Closing

The following articles were helpful during development — thank you for the clear explanations!