node-ezunpaywall

Tools for ezunpaywall

Table of content

• Prerequisites
• Installation
• Development
• Commands
  • config
  • ping
  • update-job-file
  • update-job-period
  • update-job-snapshot
  • update-report
  • update-status
  • enrich
  • apikey-create
  • apikey-update
  • apikey-delete
  • apikey-get
• Test

Prerequisites

The tools you need to let node-ezunpaywall run are :

• git >= 2.27.0
• npm >= 6.14.8
• NodeJS >= 14.15.0

Installation

$ git clone https://github.com/ezpaarse-project/node-ezunpaywall.git
$ cd node-ezunaywall
$ npm i -g .

Development

$ git clone https://github.com/ezpaarse-project/node-ezunpaywall.git
$ cd node-ezunpaywall
$ npm i

Commands

You can get help for any command by typing ezu <command> --help.

The module provides an ezunpaywall command (aliased ezu).

Name	Description
config [options]	config management command to establish the connection between the command and ezunpaywall
ping	check if services are available
update-job-file [options]	insert the content of changefile installed on ezunpaywall
update-job-period [options]	start an unpaywall data update process
update-job-snapshot [options]	download and insert the current snapshot
update-report [options]	get report of update process
update-status [options]	get status of update process
enrich [options]	enrich file with unpaywall attributes
apikey-create [options]	create new apikey
apikey-update [options]	update apikey
apikey-delete [options]	delete apikey
apikey-get [options]	get config of apikey
help [command]	display help for command

config

Manage config to fetch ezunpyawall.

Parameters

Name	Description
--get	display the configuration
--set <key> <value>	update config
-L --list	list of attributes required for configuration
-h, --help	display help for command

Examples

$ ezunpaywall config -L

baseURL
apikey
apikey

ezunpaywall config --set baseURL https://localhost.test.fr

{
  "baseURL": "https://localhost.test.fr",
  "apikey": "changeme",
}
info: from /home/user/.config/ezunpaywall.json

ping

Check if services are available.

Parameters

Name	Description
-u --use	Use a custom config

Example

ezunpaywall ping

info: Ping graphql service: OK
info: Ping update service: OK
info: Ping enrich service: OK
info: Ping apikey service: OK
info: ezmeta: OK
info: You have access to graphql, enrich, update service(s)

---

update-job-file

insert the content of changefile installed on ezunpaywall

Parameters

Name	Description
--file	snapshot's file installed on ezunpaywall
--offset	line where processing will start
--limit	line where processing will end
-L --list	list of snapshot installed on ezunpaywall
-I --index	name of the index to which the data is inserted
-h, --help	display help for command

Example

ezu update-job-file --file fake1.jsonl.gz

info: Insert "fake1.jsonl.gz"

---

update-job-period

start an unpaywall data update process

Parameters

Name	Description
--force	force update without check if is already installed
--startDate <startDate>	start date to download and insert updates from unpaywall
--endDate <endDate>	end date to download and insert updates from unpaywall
--interval <interval>	interval of update (day or week)
-L --list	list of snapshot installed on ezunpaywall
-I --index	name of the index to which the data is inserted

ezu update-job-period --startDate 2021-12-01 --endDate 2021-12-07

info: Insert "day" changefiles between "2021-12-01" and "2021-12-07"

ezu update-job-period --period week --startDate 2021-12-01 --endDate 2021-12-07

info: Insert "week" changefiles between "2021-12-01" and "2021-12-07"

---

update-job-snapshot

download and insert the current snapshot

Parameters

Name	Description
-I --index	name of the index to which the data is inserted

ezu update-job-snapshot

info: Insert current snapshot

---

update-report

get report of update process

Parameters

Name	Description
--file <interval>	changefile installed on ezunpaywall
--latest	latest report
-L --list	list of snapshot installed on ezunpaywall
-I --index	name of the index to which the data is inserted

Examples

ezu update report -L

? reports (Use arrow keys)
❯ report1.json
  report2.json
  report3.json

{
  "done": true,
  "createdAt": "2021-07-23T08:13:40.802Z",
  "endAt": "2021-07-23T08:13:40.902Z",
  "steps": [
    {
      "task": "insert",
      "file": "fake1.jsonl.gz",
      "linesRead": 50,
      "percent": 100,
      "took": 0.084,
      "status": "success"
    }
  ],
  "error": false,
  "took": 0.100
}

ezu update report --latest

{
  "done": true,
  "createdAt": "2021-07-23T08:13:40.802Z",
  "endAt": "2021-07-23T08:13:40.902Z",
  "steps": [
    {
      "task": "insert",
      "file": "fake1.jsonl.gz",
      "linesRead": 50,
      "percent": 100,
      "took": 0.084,
      "status": "success"
    }
  ],
  "error": false,
  "took": 0.100
}

---

update-status

get status of update process

Parameters

Name	Description
--verbose	show with load bard

Example

ezu update status

info: An update is being done
{
  "state": {
    "done": false,
    "createdAt": "2021-07-23T08:13:38.334Z",
    "endAt": null,
    "steps": [
      {
        "task": "getChangefiles",
        "took": 0.012,
        "status": "success"
      },
      {
        "task": "insert",
        "file": "fake3.jsonl.gz",
        "linesRead": 0,
        "percent": 0,
        "took": 0,
        "status": "inProgress"
      }
    ],
    "error": false
  }
}

or

ezu update status

info: No update is in progress
info: Use ezu update report --latest to see the latest report

---

enrich

enrich file with unpaywall attributes

Parameters

Name	Description
--file <file>	file wich must be enriched
--separator	separator of csv file
--attributes	attributes which must be enriched in graphql format. By default all attributes are added
--out	name of enriched file. By default, the output file is named: out.jsonl / out.csv
--verbose	display loadbar and exit if process end
-I --index	name of the index to which the data is inserted

Examples

ezu enrich job --file mustBeEnrich.csv --separator ";"

ezu enrich job --file mustBeEnrich.jsonl --separator ";" --attributes "{ is_oa, best_oa_location { license }, z_authors{ family } }"

---

apikey-create

create new apikey

Parameters

Name	Description
--keyname	name of apikey
--access	name of access services of apikey seperated by comma. By default it set at ['graphql']
--attributes	unpaywall attributes seperated apikey seperated by comma. By default it set at '*'
--allowed	indicates if the key is authorized or not. "true" or "false" only. By default it set at true

Example

ezu apikey-create --keyname user1

{
  "apikey": "abcd1",
  "config": {
    "name": "user1",
    "access": [
      "graphql"
    ],
    "attributes": ["*"],
    "allowed": true
  }
}

ezu apikey-create --keyname user2 --access graphql,enrich,update --allowed false --attributes doi,is_oa

{
  "apikey": "abcd2",
  "config": {
    "name": "user2",
    "access": [
      "graphql",
      "enrich",
      "update"
    ],
    "attributes": ["doi", "is_oa"],
    "allowed": false
  }
}

---

apikey-update

update apikey

Parameters

Name	Description
--apikey	apikey
--keyname	name of apikey
--access	name of access services of apikey seperated by comma. By default it set at ['graphql']
--attributes	unpaywall attributes seperated apikey seperated by comma. By default it set at '*'
--allowed	indicates if the key is authorized or not. "true" or "false" only. By default it set at true

Example

ezu apikey-update --apikey demo --keyname updated-demo --access graphql --attributes doi,is_oa

{
  "apikey": "demo",
  "config": {
    "name": "updated-demo",
    "access": [
      "graphql"
    ],
    "attributes": ["doi","is_oa"],
    "allowed": true
  }
}

---

apikey-delete

delete apikey

Name	Description
--apikey	apikey

Example

apikey-delete --apikey demo

info: apikey [demo] is deleted successfully

---

apikey-get

get config of apikey

Parameters

Name	Description
--apikey	apikey
--all	get all apikey

Example

ezu apikey-get --apikey demo

{
  "name": "user3",
  "access": [
    "enrich",
    "graphql"
  ],
  "attributes": "*",
  "allowed": true
}

apikey-load

load the content of JSON file of apikey

Parameters

Name	Description
--file	filepath of JSON file of apikey

Example

ezu apikey-load --file ./keys.json

info: Your apikey file are loaded successfully

Unpaywall structure

To see all available unpaywall attributes, click here.

Test

Make sure you have ezunpaywall start in dev mode

$ npm run test