through2
    Overview
    Documentation
    Insights
    Code
    Contributors
    Dependencies
    Alternatives

through2

A tiny wrapper around Node.js streams.Transform (Streams2/3) to avoid explicit subclassing noise

3.0.1  •  Published 7 months ago  •  by Rod Vagg  •  MIT License

NPM

A tiny wrapper around Node.js streams.Transform (Streams2/3) to avoid explicit subclassing noise

Inspired by Dominic Tarr’s through in that it’s so much easier to make a stream out of a function than it is to set up the prototype chain properly: through(function (chunk) { ... }).

Note: Users of Node.js 0.10 and 0.12 should install through2@2.x. As of through2@3.x, readable-stream@3 is being used and is not compatible with older versions of Node.js. v2.x support is being maintained on the v2.x branch.

fs.createReadStream('ex.txt')
  .pipe(through2(function (chunk, enc, callback) {
    for (var i = 0; i < chunk.length; i++)
      if (chunk[i] == 97)
        chunk[i] = 122 // swap 'a' for 'z'

    this.push(chunk)

    callback()
   }))
  .pipe(fs.createWriteStream('out.txt'))
  .on('finish', () => doSomethingSpecial())

Or object streams:

var all = []

fs.createReadStream('data.csv')
  .pipe(csv2())
  .pipe(through2.obj(function (chunk, enc, callback) {
    var data = {
        name    : chunk[0]
      , address : chunk[3]
      , phone   : chunk[10]
    }
    this.push(data)

    callback()
  }))
  .on('data', (data) => {
    all.push(data)
  })
  .on('end', () => {
    doSomethingSpecial(all)
  })

Note that through2.obj(fn) is a convenience wrapper around through2({ objectMode: true }, fn).

API

through2([ options, ] [ transformFunction ] [, flushFunction ])

Consult the stream.Transform documentation for the exact rules of the transformFunction (i.e. this._transform) and the optional flushFunction (i.e. this._flush).

options

The options argument is optional and is passed straight through to stream.Transform. So you can use objectMode:true if you are processing non-binary streams (or just use through2.obj()).

The options argument is first, unlike standard convention, because if I’m passing in an anonymous function then I’d prefer for the options argument to not get lost at the end of the call:

fs.createReadStream('/tmp/important.dat')
  .pipe(through2({ objectMode: true, allowHalfOpen: false },
    (chunk, enc, cb) => {
      cb(null, 'wut?') // note we can use the second argument on the callback
                       // to provide data as an alternative to this.push('wut?')
    }
  )
  .pipe(fs.createWriteStream('/tmp/wut.txt'))

transformFunction

The transformFunction must have the following signature: function (chunk, encoding, callback) {}. A minimal implementation should call the callback function to indicate that the transformation is done, even if that transformation means discarding the chunk.

To queue a new chunk, call this.push(chunk)—this can be called as many times as required before the callback() if you have multiple pieces to send on.

Alternatively, you may use callback(err, chunk) as shorthand for emitting a single chunk or an error.

If you do not provide a transformFunction then you will get a simple pass-through stream.

flushFunction

The optional flushFunction is provided as the last argument (2nd or 3rd, depending on whether you’ve supplied options) is called just prior to the stream ending. Can be used to finish up any processing that may be in progress.

fs.createReadStream('/tmp/important.dat')
  .pipe(through2(
    (chunk, enc, cb) => cb(null, chunk), // transform is a noop
    function (cb) { // flush function
      this.push('tacking on an extra buffer to the end');
      cb();
    }
  ))
  .pipe(fs.createWriteStream('/tmp/wut.txt'));

through2.ctor([ options, ] transformFunction[, flushFunction ])

Instead of returning a stream.Transform instance, through2.ctor() returns a constructor for a custom Transform. This is useful when you want to use the same transform logic in multiple instances.

var FToC = through2.ctor({objectMode: true}, function (record, encoding, callback) {
  if (record.temp != null && record.unit == "F") {
    record.temp = ( ( record.temp - 32 ) * 5 ) / 9
    record.unit = "C"
  }
  this.push(record)
  callback()
})

// Create instances of FToC like so:
var converter = new FToC()
// Or:
var converter = FToC()
// Or specify/override options when you instantiate, if you prefer:
var converter = FToC({objectMode: true})

See Also

License

through2 is Copyright © Rod Vagg and additional contributors and licensed under the MIT license. All rights not explicitly granted in the MIT license are reserved. See the included LICENSE file for more details.

We are hiring frontend and backend developers!
Come and help us build the ultimate open source directory 🔥

Popularity

Weekly Downloads
13.8M
Stars
1.6K

Maintenance

Development

Last ver 7 months ago
Created 6 years ago
Last commit 7 months ago
19 days between commits

Technology

Node version: 11.10.1
17.4K unpacked

Legal and Compliance

MIT License
OSI Approved
0 vulnerabilities

Contributors

17 contributors
Rod Vagg
Maintainer, 74 commits, 13 merges, 6 PRs
Works at require.io
Bryce Baril
Maintainer, 11 commits, 1 merges, 1 PRs
Works at NodeSource
Mathias Buus
Maintainer, 7 commits, 2 merges, 5 PRs
Works at hyperdivision
Delapouite
Maintainer, 3 commits, 2 merges, 1 PRs
Works at Powipu
Dane Stuckel
3 commits, 1 PRs
Works at WeWork
Eric Schoffstall
2 commits, 2 PRs
Works at staeco

Tags

stream
streams2
through
transform
© 2019 Devstore, Inc.
Devstore helps developers find and use open-source packages, so they can focus on building amazing things