[PP-Script] NZBGet-Elasticsearch

Share your scripts or request scripts with specific features.
Forum rules
Please keep the forum clean - one topic per script. Questions not related to a specific script should be posted in Support forum.
Post Reply
darkciti
Posts: 2
Joined: 10 Jun 2018, 22:37

[PP-Script] NZBGet-Elasticsearch

Post by darkciti » 10 Jun 2018, 23:05

Hey everyone,

I've put together a very simple (at this point) script that inserts a download record into Elasticsearch. I plan to add functionality, but this is very much version 0.10. I welcome any feedback and/or pull requests if you know python and the Elastic Stack.

Usage is simple, just set the ElasticSearch host and port.

Once the data is in Elasticsearch, you can create dashboards or visualizations with Kibana, or you can point Grafana at it if you want fancier dashboards.

There are other ways you can get nzbget logs into Elasticsearch, like using Logstash Grok Filters, but I just wanted something quick and simple.

You can check it out here: https://github.com/chrisbergeron/nzbget-elasticsearch

Code: Select all

#!/usr/bin/env python
#
# Insert a document(record) into ElasticSearch upon completion
#

##############################################################################
### NZBGET POST-PROCESSING SCRIPT                                          ###

# Log output to ElasticSearch
# Created by Chris Bergeron <nzbget@chrisbergeron.com>
#
# Inserts NZBGet logs into ElasticSearch.  Options are hostname and port.
# See https://chrisbergeron.com/ for details
#
# NOTE: This script requires Python to be installed on your system and the
# Elasticsearch python module (pip install elasticsearch)
# Visit:  https://chrisbergeron.com/ for more information

##############################################################################
### OPTIONS                                                                ###

# ElasticSearch server host
#Host=localhost

# ElasticSearch port (default is 9200); uncomment if different from default
#Port=9200

# NZB Disposition (the outcome of the download attempt)
#Disposition=Download Complete

### NZBGET POST-PROCESSING SCRIPT
##############################################################################

import os,datetime,sys,uuid
from datetime import datetime
import elasticsearch
from elasticsearch import Elasticsearch

# Exit codes used by NZBGet
POSTPROCESS_SUCCESS=93
POSTPROCESS_ERROR=94
POSTPROCESS_NONE=95

# Check if the script is called from nzbget 15.0 or later
if not 'NZBOP_NZBLOG' in os.environ:
    print('*** NZBGet post-processing script ***')
    print('This script is supposed to be called from nzbget (15.0 or later).')
    sys.exit(POSTPROCESS_ERROR)

# Make sure all our options are set (Settings -> ESLog)
required_options = ('NZBPO_HOST', 'NZBPO_PORT', 'NZBPO_DISPOSITION', 'NZBPP_NZBNAME')
for	optname in required_options:
	if (not optname in os.environ):
		print('[ERROR] Option %s is missing in configuration file. Please check script settings' % optname[6:])
		sys.exit(POSTPROCESS_ERROR)

# Get environment variables/options from NZBGet
id = uuid.uuid4()
nzbname = os.environ['NZBPP_NZBNAME']
host = os.environ['NZBPO_HOST']
port = os.environ['NZBPO_PORT']
disposition = os.environ['NZBPO_DISPOSITION']

# Create ES server object
try:
  es = Elasticsearch([{'host': host, 'port': port}])
  # print('Connected ', es.info())
except Exception as ex:
  print ('Error: ', ex)

# Replace periods with spaces in our filename log entry
nzbname=nzbname.replace('.', ' ')
timestamp=datetime.now()

# Insert our record into ES
try:
  es = Elasticsearch([{'host': host, 'port': port}])

  doc = {
    'nzbname': nzbname,
    'disposition': disposition,
    'timestamp': timestamp,
  }
  res = es.index(index="nzbget", doc_type='document', id=id, body=doc)
  # print(res['result'])

except Exception as ex:
  print ('Couldnt Insert: ', ex)

print ("[DETAIL] Inserting record into ElasticSearch: ", disposition, " ", nzbname, " ", timestamp)
sys.stdout.flush()

# All OK, returning exit status 'POSTPROCESS_SUCCESS' (int <93>) to let NZBGet know
# that our script has successfully completed.
sys.exit(POSTPROCESS_SUCCESS)

hugbug
Developer & Admin
Posts: 7027
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: [PP-Script] NZBGet-Elasticsearch

Post by hugbug » 14 Jun 2018, 18:11

Thanks for sharing the script.

A tip:

Code: Select all

print ("[DETAIL] Inserting record into ElasticSearch: ", disposition, " ", nzbname, " ", timestamp)
Using of commas makes python to print all arguments as a list. That's probably isn't what you wanted. Try `+` instead or other formatting functions producing one string instead of a list.

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest