url-shortener-service

Getting Started

Functions deployed with Serverless framework to AWS Lambda at

https://7b8kyoyw1e.execute-api.us-east-1.amazonaws.com/prod

Data storage - MongoDB Atlas. Build with TypeScript.

Postman collection can be used to test endpoints locally and remotely.

function	type	path	execution time	description
createShortUrlByHash	http	POST /hash	-//-	Request a shortened url. Even if a url was already requested it should generate a new hash.
getUrlByHash	http	GET /url	-//-	Get the url by using hash.
getStatsByUrl	http	GET /stats/url	-//-	Get the statistics of a url.List of all hashes which were generated and list of ip addresses of users.
cleanup	schedule	-//-	0 0 * * ? *	Cronjob which will delete every day at 12.00am hashes of URLs which are not longer used by 12 months.

The hash is made of 8 characters using charset with 64 elements, meaning there are 64^8 combinations. It is used as an _id in MongoDB schema, so after "cleanup" job hash becomes available again

TODO:

Some things would be great to add:

• use base62 instead of base64 encoding for url hash. Additional characters $@ are not "friendly" for short-url;
• add black list of urls, during creation check if url is not blacklisted or using bloom filter;
• use cache with ttl for entries in front of database;
• alerting in slack channel;
• based on usage of GET /stats/url data can be stored in HDFS or similar to enable stats being aggregated;
• setup authorization for GET /stats/url, make it private;

Prerequisites

Minimal requirements to set up the project:

• Node.js v12, installation instructions can be found on the official website, a recommended installation option is to use Node Version Manager. It can be installed in a few commands.
• A package manager npm. All instructions in the documentation will follow the npm syntax.
• Serverless
• docker-compose
• Optionally, a Git client.

Installing

Start by cloning the repository:

git clone git@github.com:oleg-koval/brevis.git

In case you don't have a git client, you can get the latest version directly by using this link and extracting the downloaded archive.

Go the the right directory and install dependencies:

cd brevis
npm install

Install serverless:

npm install -g serverless

Login to serverless:

sls login

That's it! You can now go to the next step.

Run locally

Environment is managed with dotenv. Rename .env.example to .env.

Environment variable MONGODB_CONNECTION_STRING should be present: f.e.: mongodb://0.0.0.0:27017/test

Run mongodb container in detached mode:

docker-compose up -d

Run tests with coverage:

npm run test:coverage

Alternatively serverless offline can be used to run functions locally (connection to local MongoDB or hosted should be provided):

sls offline start

It is possible to invoke functions locally (connection to local MongoDB or hosted should be provided) with sls invoke local --function <functionName>

To shutdown database without delete all containers.

docker-compose stop

To shutdown database and delete all containers.

docker-compose down

Run tests

Tests

All tests are being executed using Jest. All tests files live side-to-side with a source code and have a common suffix: .spec.ts. Some helper methods are being stored in the test directory.

There are three helper scripts to run tests in the most common scenarios:

npm run test
npm run test:watch
npm run test:coverage

Continuous Integration / Delivery

GitHub Actions are configured and run the tests, linting, formatting, spellcheck, types whenever a commit is pushed to this repository master or any other branch.

Also used to deploy automatically to AWS Lambda production after PR merged to master.

Formatting

This project uses Prettier to automate formatting.

npm run format
npm run format:fix

Linting

This project uses ESLint to enable static analysis.

npm run lint
npm run lint:fix

Automation

• GitHub Actions
• Dependabot
• Serverless