Creating processes in Node allows parallelization

While threads of execution can be created easily in node by spinning off async operations from the main thread, if you want to take advantage of a multiple cores you will need to use some form of child process. My use for this was specifically when working with large data sets that would take extensive time to process. I used streams (Stream howto) and parallel processing. The streams process the data piecemeal, but that just makes it possible, to actually speed things up I needed to break the program up into multiple processes with each one utilizing streams.

child_process contains the modules used for creating sub processes

spawn, exec, fork are the main ones.

Creating child node processes with child_process.fork()

This module allows creating child node processes so you can run node code in a separate process.
cluster.fork() wraps this module
You can also use the wrapper of child_process.fork() provided by cluster.fork(), which I will demonstrate later.

Using fork()

Passing command line arguments The array ([‘list of’, ‘arguments’]) passes args available to the child in process.argv. execArgv passes args to the node instance running the sub process. In this case I am passing debug configuration so I debug the subp. The input object is to pass custom information. It is accessed through process.input or any name you choose, input was just an example.

const { fork } = require('child_process')
const subp = fork('./testchild.js', ['list of', 'arguments'], {
  execArgv: ['--inspect-brk=9332'],
  input: { filename: 'sourcefile.txt' }, 
});

Sending message to, recieving them from the child process

  //send message to the child process
  const subp = fork('./writetodb.js')
  subp.send({ writelines });   
  // listen for messages from forked process
  subp.on('message', (mesage) => {
    log.info(`Lines to write: ${message.writelines}`);
  });

Listening, Sending info back from the child

  process.on('message', (msg) => {
    const rslt = operateon(msg)
    process.send(rslt);
  });

Using cluster.fork()

Instead of calling a separate script, the sript runs first as a master then as worker, and you set up different content to run based on this.

if (cluster.isMaster) {  
  const cpus = os.cpus().length;  
  console.log(`Forking for ${cpus} CPUs`);  
  // run anything you need as the master  
  masterFcn()  
  for (let i = 0; i<cpus; i++) {  
    cluster.fork();  
  }
} else {  
  // For each fork(), this part will run  
  workerFcn();   
}

Debug ports should automatically work with cluster.fork()

By default it should increment the port for each child process so you can access them all. Ex: if the master was 9330, children would be 9331, 9332, 9333 …
The port can also be configured
cluster.setupMaster({execArgv: [“—inspect=6001”]})

What this means is that you can pass messages to the server, and they will be distributed the the children. Specifically it would act like a load balancer, so each message would get sent to one child.
You could set up a multi-core express server this way
Each child would run the same code, but the requests would be distributed between them. So this is good if the child processes are duplicates.

If you don’t need a node process; spawn or exec

These child processes are for running something other than node scripts. Each has certain features that I will lay out in a basic way.

spawn

Launch a command in a new process. Implements the EventEmitter API so you can register handlers just like with a stream.
It uses streams for in/out
The strengths of spawn is that it uses streams for stdin/stdout. It does not spawn a shell, so slightly lower overhead.
Basic setup of spawn

const { spawn } = require('child_process')
const childproc = spawn('cmd')
childproc.on('eventname', () => {})

Events include: exit, disconnect, error, close, message Spawn processes also have the three std io streams
childproc.stdin/stdout/stderr. You can read from stdout/stderr - Meaning that the master can read from these since they are what the child outputs to. writing is done to stdin - The master can write to stdin.
Some examples of connecting up stdin/stdout

// How to pass cli options
const tr_braces = spawn('tr', ['{}[]()' '[](){}']);  
const sed_space = spawn('sed', ['-r', 's/\s+/ /g']);  
// Set up a pipe to send its output to the stdin of the child proc.
readable.pipe(tr_braces.stdin)
// Have one pipe to another
tr_braces.stdout.pipe(sed_space.stdin)

exec

This command does spawn a shell, and buffers all the output and passes it all at once via a callback. This makes it simple to use but not very good for large amounts of data. Note you pass shell syntax directly, so dynamic input could be dangerous.
The output is that of the whole command
Unlike spawn where you set up in and out streams, with exec you get the buffered result of the whole command.
exec to run a generic shell command -
This spawns a shell and buffers all the output such that you handle it with a callback. The command is handled as a single statement.

const shellCommand = comm => {  
  return new Promise((resolve, reject) => {  
    exec(comm, (error, stdout, stderr) => {  
      if(error){  
        reject(error)  
      }else{  
        resolve({stdout: stdout, stderr: stderr})  
      }
    }) 
  })
}

Didn’t I just say not to take dynamic input

So I did. What I really meant was, don’t take unknown input, such as from user interaction. As a wrapper to run shell commands of your design, should be completely fine.

child_process allows running separate processes from node

Whether you need to run node code separately to take advantage of processing power, other code for the same reason, or just want to run a shell command, child_process is the way to go about doing this.