Node.js Basics
This is a reading note for node.js part on tutorialspoint.
Introduction
Node.js (MIT License) is a server-side framework built on Good V8 Engine. From Tutorialpoints.com, it has following features:
-
Asynchronous and Event Driven: the server will always move on. Once it gets a notification of a event with returned data, it will trigger this event.
-
Single Threaded Model but Highly Scalable: although node.js use a single threaded server, it can response to multiple requests because of its asynchronous feature. It is easy to be scaled.
-
Node.js does not buffer any data.
Components
One node.js application has following three parts:
Import Required Modules: use require
to import node.js modules. Example: var http = require("http")
Create server: user http.createServer()
to create server instance. This method includes callbacks for response information, such as port number, content-type, etc:
var http = require('http'); http.createServer(function(request, response){ response.writeHead(200,{'Content-Type': 'text/plain'}); response.add('Hello World\n'); //add data in response body }).listen(8081);//in example use 8081 as port number
Start Server: use node xxx.js
in terminal to start server. Use name of server js file. Example: node config.js
.
NPM
node.js use npm
(Node Package Manager) to install/control node modules. npm install package_name [-g] [--save/--save-dev]
can be used to install package (--save
or --save-dev
is used to add dependencies to your package.json file). There are two way of installation: Global and Local:
-
Local modules are installed inside project, in
node_modules
folder. Locally deployed packages are accessiable viarequire()
method. Local packages can be listed bynpm ls
command. -
Globally packages are stored in system directory. It cannot be imported by
require()
. They can be used in terminal.npm ls -g
can be used to list global packages.
npm also has following common command:
-
npm uninstall package_name
: uninstall a package -
npm update package_name
: update a package -
npm search package_name
: search package name using npm.
package.json
For project: Run npm init
at top level of a project can be used to generate package.json
for this project, which is a JSON file defining the property of this project. A few questions will be asked during initialization process.
A package.json
for project contains dependencies
, which is for publication pages, and devDependencies
for dev/test only packages. Following is an example:
{ "name": "test", //name of project "version": "1.0.0", "description": "", "main": "gulpfile.js", //entry point of project "scripts": { //handle status of project, see https://docs.npmjs.com/misc/scripts for detail "test": "echo \"Error: no test specified\" && exit 1" }, "author": "test author", "license": "ISC", //packages and versions for dev/test only "devDependencies": { "gulp": "^3.9.1" }, //packages and versions for production "dependencies": { "gulp": "^3.9.1" } }
For node modules: there is also a package.json file in each node module in node_modules
folder. Following is the package.json
file in gulp-jade
module package:
{ "name": "gulp-jade", "version": "1.1.0", "description": "Compile Jade templates", "keywords": [ "jade", "gulpplugin", "stream", "compile" ], "repository": { "type": "git", "url": "git+https://github.com/phated/gulp-jade.git" }, "author": { "name": "Blaine Bublitz", "email": "[email protected]" }, "dependencies": { "gulp-util": "^3.0.2", "jade": "1.1 - 1.11", "through2": "^2.0.0" }, "files": [ "index.js", "LICENSE" ], "engines": { "node": ">= 0.10" }, "scripts": { "test": "gulp jshint && tap ./test" }, "license": "MIT", "devDependencies": { "gulp": "^3.8.10", "tap": "^1.3.1", "gulp-jshint": "^1.9.0" }, "gitHead": "671714d09e9ccd291489f22988bbc40fe8db40ec", "bugs": { "url": "https://github.com/phated/gulp-jade/issues" }, "homepage": "https://github.com/phated/gulp-jade#readme", "_id": "[email protected]", "_shasum": "cb2f332ac46824671f36891655c165c591c33bd4", "_from": "[email protected]", "_npmVersion": "2.8.3", "_nodeVersion": "0.10.36", "_npmUser": { "name": "phated", "email": "[email protected]" }, "maintainers": [ { "name": "phated", "email": "[email protected]" } ], "dist": { "shasum": "cb2f332ac46824671f36891655c165c591c33bd4", "tarball": "https://registry.npmjs.org/gulp-jade/-/gulp-jade-1.1.0.tgz" }, "directories": {}, "_resolved": "https://registry.npmjs.org/gulp-jade/-/gulp-jade-1.1.0.tgz", "readme": "ERROR: No README data found!" }
npm adduser
to register current user to npm repository, then npm publish
can be used to publish current module to repository
address in package.json
.
Concurrency: Callback and Events
Callbacks
JavaScript callback functions are heavily used in node.js to make sure no block exists. Following is an example:
//Assume 'input.txt' has one line 'Hello World!' var fs = require('fs'); fs.readFile('input.txt', function(err,data){ if(err) return console.log(err); console.log(data.toString()); }); console.log('End Program!');
The code above should output: End Program! Hello World!
. Program goes till readFile()
finished, then callback statements will run.
Events
Node.js use observer pattern, which means there exists dependency from one to many. Many observers (i.e. event listeners) will observe one subject (i.e. event). When subjects changes, all observers will be notified and updated based on the update.
Node thread keeps an event loop and whenever any task get completed, it fires the corresponding event which signals the event listener function to get executed.
Image from tutorialspoint.com:
images/articles/2016/javascript/node_js_events.png
This is the difference between callbacks and events. The EventEmitter
is used to bind events and event listeners.
EventEmitter Methods:
-
eventemitter.on('eventName',eventhandler)
is used to bind event name and event handler. -
eventemitter.once('eventName', eventhandler)
is to add a listener that only the next time event is fired, after which it is removed. -
eventemitter.removeListener
/eventemitter.removeAllListeners
: remove listener -
eventemitter.emit('eventName')
is used to fire an event.
'eventName' above is just a identifier for an event concept, it's not a real object. See example below:
// Import events module var events = require('events'); // Create an eventEmitter object var eventEmitter = new events.EventEmitter(); // Create an event handler as follows var connectHandler = function connected() { console.log('connection succesful.'); // Fire the data_received event eventEmitter.emit('data_received'); } // Bind the connection event with the handler eventEmitter.on('connection', connectHandler); // Bind the data_received event with the anonymous function (callback) eventEmitter.on('data_received', function(err, data){ if(err) console.log('err'); else{ setTimeout(function(){ console.log('data received'); },2000); } }); // Fire the connection event eventEmitter.emit('connection'); console.log("Program Ended.");
Since I purposely added a setTimeout
function to test callback, code above will generate output:
connection succesful. Program Ended. data received
If no setTimeout
(i.e. data_received
event finish immediately), 'data received' will output before 'Program Ended'.
In Node Application, any async function accepts a callback as a last parameter and the callback function accepts error as a first parameter.
Streams
Reading/Writing files needs require('fs')
module.
Streams is used to read/write data. In node.js, there are four type of streams:
-
Readable: stream for read operation. Common operation:
fs.createReadStream(file)
-
Writable: stream for write operation. Common operation:
fs.createWriteStream(file)
,stream.write(data)
,stream.end()
-
Duplex: stream for both read and write operation.
-
Transform: a duplex stream which output is based on input
Each type of stream is an EventEmitter
instance. This means it is used the same way as EventEmitter above. Some common events that are bind to these EventEmitters are:
-
data: fired when data is available to read
-
end: fired when no more data to be read
-
error: fired when getting error while receiving/writing data
-
finish: fired when transferring data process is finished
Piping
is a mechanism where the output of one stream is the input of another stream. Piping stream in node.js looks like following:
var fs = require('fs'); var readerStream = fs.createReadStream('input.txt'); var writeStream = fs.createWriteStream('output.txt'); //Pipe Operation. Read input and write to output readerStream.pipe(writeStream);
Chanining
is a mechanism to connect output of one stream to another stream and create a chain of multiple stream operations.
It usually use with Piping stream. Example:
var fs = require('fs'); var zlib = require('zlib'); fs.createReadStream('input.txt') .pipe(zlib.createGzip()) .pipe(fs.createWriteStream('input.txt.gz'));
Global Variables and Common Modules in node.js
Following are some global variables in node.js, which can be used directly, instead of requiring it first:
-
_filename
: the filename of the code being executed. Example:console.log(_filename)
-
_dirname
: name of the directory that the currently executing script resides in -
setTimeout(function, milliseconds)
/clearTimeout(object)
/setInterval(function, milliseconds)
: timer functions in node.js.
Two global objects: console
and process
can be used directly in js file. console
is used to print information. process
is used to get information on current process.
Following modules are commonly used in node.js. They needs to be required at the beginning:
-
OS Module:
require('os')
module provides a few basic operating system related utility functions -
Path Module:
require('path')
module provides methods to handle file paths. -
Net Module:
require('net')
module is used to create both servers and clients. It provides an asynchronous network wrapper. It has methods such asnet.createServer()
,net.connect()
, etc. See here for details. -
DNS Module: Provides functions to do DNS lookup. Provides functions to use underlying operating system name resolution functions. It provides an asynchronous network wrapper. It has methods such as
dns.lookup()
,dns.resolve()
, etc. See here for details. -
Domain Module:
require('domain')
module is used to intercept unhandled error:
There are two ways to handle error: Internal Binding
, meaning error emmitter is executing its code within run
method of a domain; External Binding
, meaning error emmitter is added explicitly to a domain using its add method.
Error Object
Reference to this site: in node.js, error
is a build-in object that provides a standard set of useful information when an error occurs. such as a stack trace and the error message:
var error = new Error('The error message'); console.log(error);
Output will be:
{ stack: [Getter/Setter], arguments: undefined, type: undefined, message: 'The error message' }
domain.create()
needs to be used create an instance of domain, then it can be used to listen to error event:
var EventEmitter = require('events').EventEmitter; var domain = require('domain'); var emitter1 = new EventEmitter(); //create an instance of eventemitter var domain1 = domain.create(); //create a domain //Domain handle error domain1.on('error',function(err){ console.log('domain1 error handler'); }); domain1.add(emitter1); //Explicit Binding //Bind error event and error handler emitter1.on('error',function(err){ console.log('emittter1 error handler'); }); //In output, fire emitter handler instead of domain handler emitter1.emit('error',new Error('Error to be handled by listener')); var domain2 = domain.create(); domain2.on('error',function(err){ console.log('domain2 error handler'); }); //Implicit Binding. Create EventEmitter inside run method domain2.run(function(){ var emitter2 = new EventEmitter(); emitter2.emit('error', new Error('Implicit error handler')); //fire an error event with error message }); //Remove explict binding and try again. Remember to remove all listeners and remove eventemitter together! emitter1.removeAllListeners('error'); domain1.remove(emitter1); emitter1.emit('error', new Error('Converted to exception. System will crash!'));
Code above will have following output. Noted error message does not show unless there is no error handler for it:
emittter1 error handler domain2 error handler events.js:72 throw er; // Unhandled 'error' event ^ Error: Converted to exception. System will crash! at Object. (/web/com/1464273829_135671/main.js:36:28) at Module._compile (module.js:456:26) at Object.Module._extensions..js (module.js:474:10) at Module.load (module.js:356:32) at Function.Module._load (module.js:312:12) at Function.Module.runMain (module.js:497:10) at startup (node.js:119:16) at node.js:929:3
Node.js as Web Server/Client
Following is the image to describe web application architecture in tutorialspoint.com:
Following code is to use node.js as web server (server.js
):
var http = require('http'); var fs = require('fs'); var url = require('url'); // Create a server http.createServer( function (request, response) { // Parse the request containing file name var pathname = url.parse(request.url).pathname; // /index.htm here // Print the name of the file for which request is made. console.log("Request for " + pathname + " received."); /* Read the requested file content from file system. Here file is index.htm, so content type is text/html. Notice the usage of file system here. */ fs.readFile(pathname.substr(1), function (err, data) { if (err) { console.log(err); // HTTP Status: 404 : NOT FOUND // Content Type: text/plain response.writeHead(404, {'Content-Type': 'text/html'}); }else{ //Page found // HTTP Status: 200 : OK // Content Type: text/plain response.writeHead(200, {'Content-Type': 'text/html'}); // Write the content of the file to response body response.write(data.toString()); } // Send the response body response.end(); }); }).listen(8081); // Console will print the message console.log('Server running at http://127.0.0.1:8081/');
Following code is to use node.js as web client (client.js
). http
module is also used in this part:
var http = require('http'); //Options to used by request var options = { host: 'localhost', port: '8081', path: '/index.htm' }; //Deal with response from web server var callback = function(response){ var body = ''; //While data is being received by client, update body variable response.on('data', function(data){ body += data; }); //Finish data receiving process response.on('end', function(){ console.log(body); }); } //Make request to web server var req = http.request(options, callback); req.end();//Close request after finishing
Following is a base index.htm
:
<html> <head> <title>Sample Page</title> </head> <body> Hello World! </body> </html>
To run code above, create these three files:
-
Then open one terminal and type
node server.js
. Terminal will show 'Server running at http://127.0.0.1:8081/' and you can open 'http://127.0.0.1:8081/' in web browser. -
Now browse 'http://127.0.0.1:8081/index.htm'. Browser should have html available and terminal will give message 'Request for /index.htm received.'.
-
Then open another terminal and type
node client.js
. New terminal should show html code (i.e. body variable). Old terminal should show 'Request for /index.htm received.' -
If you don't have
index.htm
file available, on step 2 terminal will give out error message, which is the code handled in server.js (404 part)
Express
Express is a popular Node.js framework. It has following features:
-
Set up middleware to respond to HTTP requests
-
Define the route table to perform different action based on HTTP method and url
-
Dynamically render HTML based on passing arguments to templates
Express is normally being installed with following modules (using npm install xxx --save)
:
-
body-parser: the middle-ware for handling JSON, Raw, Text and URL encoded form data.
-
cookie-parser: Parse cookie header and populate
req.cookies
with an object keyed by the cookie names -
multer - the middle-ware for handling multipart/form-data.
Routing
Express uses request
and response
objects in a callback function. Basic usage is following:
app.get('/', function(req,res){ //deal with request and response objects });
Basic way to routing (web application response to different url end point to perform operations like get, post) is following:
var express = require('express'); var app = express(); //GET for abcd, abcxd, ab123cd and so on, using Regular Expression app.get('/abc*d', function(req,res){ console.log('GET'); res.send('Hello GET'); //send response back }); //POST app.post('/',function(req,res){ console.log('POST from homepage'); res.send('Hello POST'); }); //PUT app.put('/put',function(req,res){ res.send('PUT from /put page'); }); //DELETE app.delete('/delete', function(req, res){ console.log('DELETE from /delete page'); res.send('Hello DELETE'); }); var server = app.listen(8081, function(){ console.log(server.address()); });
The console information for server.address
may return address
attribute as ::
. This is because if the host name is omitted, the IPv6 address will show ::
and IPv4 address will return 0.0.0.0
. See this Stack Overflow post for detail.
Static Files in Express
express.static
middleware is used to serve static files, such as images, stylesheets or JS files. An example is following:
/* now files in public folder (based on root) can be detected in browser */ app.use(express.static('public')); app.get('/',function(req,res){ //continue with other operations });
After running code above, URL such as http://localhost:8081/images/test.png
can be typed and viewed in browser, as long as image/test.png
is under public
folder and public
folder is in root.
Cookie
Use cookie-parser
middleware to print out cookie information. Example:
var express = require('express'); var cookieParser = require('cookie-parser'); var app = express(); app.use(cookieParser()); app.get('/', function(req, res) { console.log("Cookies: ", req.cookies); }) app.listen(8081);