version 0.4: simpler config, simpler usage

and simpler code

README.rst

@@ -6,6 +6,7 @@ Dependency
 ==========
 - Python 3
 - Tornado
+- Optional pycurl
 - All commands used in your version source files
 
 Running
@@ -16,21 +17,21 @@ To see available options::
 
 Run with one or more software version source files::
 
-  ./nvchecker source_file_1 source_file_2 ...
+  ./nvchecker source_file
 
 You normally will like to specify some "version record files"; see below.
 
 Version Record Files
 ====================
-Version record files record which version of the software you know or is available. They are simple key-value pairs of ``(name, version)`` seperated by a space\ [#]_::
+Version record files record which version of the software you know or is available. They are simple key-value pairs of ``(name, version)`` seperated by a space\ [v0.3]_::
 
   fcitx 4.2.7
   google-chrome 27.0.1453.93-200836
   vim 7.3.1024
 
-Say you've got a version record file called ``old_ver.txt`` which records all your watched software and their versions. To update it using ``nvchecker``::
+Say you've got a version record file called ``old_ver.txt`` which records all your watched software and their versions, as well as some configuration entries. To update it using ``nvchecker``::
 
-  ./nvchecker --oldver old_ver.txt --newver new_ver.txt source.ini
+  ./nvchecker source.ini
 
 Compare the two files for updates (assuming they are sorted alphabetically; files generated by ``nvchecker`` are already sorted)::
 
@@ -54,6 +55,20 @@ The software version source files are in ini format. *Section names* is the name
 
 See ``sample_source.ini`` for an example.
 
+Configuration Section
+---------------------
+A special section named ``__config__`` is special, it provides some configuration options\ [v0.4]_.
+
+Relative path are relative to the source files, and ``~`` and environmental variables are expanded.
+
+Currently supported options are:
+
+oldver
+  Specify a version record file containing the old version info.
+
+newver
+  Specify a version record file to storing the new version info.
+
 Search in a Webpage
 -------------------
 Search through a specific webpage for the version string. This type of version finding has these fields:
@@ -120,16 +135,6 @@ Other
 -----
 More to come. Send me a patch or pull request if you can't wait and have written one yourself :-)
 
-Config File
-===========
-``nvchecker`` supports a config file, which contains whatever you would give on commandline every time. This file is at ``~/.nvcheckerrc`` by default, and can be changed by the ``-c`` option. You can specify ``-c /dev/null`` to disable the default config file temporarily.
-
-A typical config file looks like this::
-
-    --oldver ~/.nvchecker/versionlist.txt --newver ~/.nvchecker/versionlist_new.txt
-
-``~`` and environmental variables will be expanded. Options given on commandline override those in a config file.
-
 Bugs
 ====
 * Finish writing results even on Ctrl-C or other interruption.
@@ -141,4 +146,5 @@ TODO
 
 Footnotes
 =========
-.. [#] Note: with nvchecker <= 0.2, there are one more colon each line. You can use ``sed -i 's/://' FILES...`` to remove them.
+.. [v0.3] Note: with nvchecker <= 0.2, there are one more colon each line. You can use ``sed -i 's/://' FILES...`` to remove them.
+.. [v0.4] This is added in version 0.4, and old command-line options are removed.

nvchecker/__init__.py

@@ -1 +1 @@
-__version__ = '0.3'
+__version__ = '0.4'

nvchecker/core.py

@@ -0,0 +1,123 @@
+# vim: se sw=2:
+
+import os
+import sys
+import logging
+import configparser
+
+from pkg_resources import parse_version
+
+from .lib import nicelogger
+from .get_version import get_version
+
+from . import __version__
+
+logger = logging.getLogger(__name__)
+
+def add_common_arguments(parser):
+  parser.add_argument('-l', '--logging',
+                      choices=('debug', 'info', 'warning', 'error'), default='info',
+                      help='logging level (default: info)')
+  parser.add_argument('-V', '--version', action='store_true',
+                      help='show version and exit')
+  parser.add_argument('file', metavar='FILE', nargs='?', type=open,
+                      help='software version source file')
+
+def process_common_arguments(args):
+  '''return True if should stop'''
+  nicelogger.enable_pretty_logging(getattr(logging, args.logging.upper()))
+
+  if args.version:
+    progname = os.path.basename(sys.argv[0])
+    print('%s v%s' % (progname, __version__))
+    return True
+
+def safe_overwrite(fname, data, *, method='write', mode='w', encoding=None):
+  # FIXME: directory has no read perm
+  # FIXME: symlinks and hard links
+  tmpname = fname + '.tmp'
+  # if not using "with", write can fail without exception
+  with open(tmpname, mode, encoding=encoding) as f:
+    getattr(f, method)(data)
+  # if the above write failed (because disk is full etc), the old data should be kept
+  os.rename(tmpname, fname)
+
+def read_verfile(file):
+  v = {}
+  with open(file) as f:
+    for l in f:
+      name, ver = l.rstrip().split(None, 1)
+      v[name] = ver
+  return v
+
+def write_verfile(file, versions):
+  # sort using only alphanums, as done by the sort command, and needed by
+  # comm command
+  data = ['%s %s\n' % item
+          for item in sorted(versions.items(), key=lambda i: (''.join(filter(str.isalnum, i[0])), i[1]))]
+  safe_overwrite(file, data, method='writelines')
+
+class Source:
+  started = False
+  tasks = 0
+  def __init__(self, file):
+    self.config = config = configparser.ConfigParser(
+      dict_type=dict, allow_no_value=True
+    )
+    self.name = file.name
+    config.read_file(file)
+    if '__config__' in config:
+      c = config['__config__']
+      d = os.path.dirname(file.name)
+      self.oldver = os.path.expandvars(os.path.expanduser(os.path.join(d, c.get('oldver'))))
+      self.newver = os.path.expandvars(os.path.expanduser(os.path.join(d, c.get('newver'))))
+
+  def check(self):
+    self.started = True
+
+    if self.oldver:
+      self.oldvers = read_verfile(self.oldver)
+    else:
+      self.oldvers = {}
+    self.curvers = self.oldvers.copy()
+
+    config = self.config
+    for name in config.sections():
+      if name == '__config__':
+        continue
+      self.task_inc()
+      get_version(name, config[name], self.print_version_update)
+
+  def task_inc(self):
+    self.tasks += 1
+
+  def task_dec(self):
+    self.tasks -= 1
+    if self.tasks == 0 and self.started:
+      if self.newver:
+        write_verfile(self.newver, self.curvers)
+      self.on_finish()
+
+  def print_version_update(self, name, version):
+    try:
+      if version is None:
+        return
+
+      oldver = self.oldvers.get(name, None)
+      if not oldver or parse_version(oldver) < parse_version(version):
+        logger.info('%s updated version %s', name, version)
+        self.curvers[name] = version
+        self.on_update(name, version, oldver)
+      else:
+        logger.debug('%s current version %s', name, version)
+    finally:
+      self.task_dec()
+
+  def on_update(self, name, version, oldver):
+    pass
+
+  def on_finish(self):
+    pass
+
+  def __repr__(self):
+    return '<Source from %r>' % self.name

nvchecker/main.py

@@ -1,104 +1,44 @@
 #!/usr/bin/env python3
 
-import configparser
 import logging
 import argparse
 
-from pkg_resources import parse_version
 from tornado.ioloop import IOLoop
 
 from .lib import notify
-
+from . import core
-from .get_version import get_version
-from . import util
 
 logger = logging.getLogger(__name__)
 notifications = []
-g_counter = 0
-g_oldver = {}
-g_curver = {}
 args = None
 
-def task_inc():
+class Source(core.Source):
-  global g_counter
+  def on_update(self, name, version, oldver):
-  g_counter += 1
+    if args.notify:
+      msg = '%s updated to version %s' % (name, version)
+      notifications.append(msg)
+      notify.update('nvchecker', '\n'.join(notifications))
 
-def task_dec():
+  def on_finish(self):
-  global g_counter
-  g_counter -= 1
-  if g_counter == 0:
     IOLoop.instance().stop()
-    write_verfile()
-
-def load_config(*files):
-  config = configparser.ConfigParser(
-    dict_type=dict, allow_no_value=True
-  )
-  for file in files:
-    with open(file) as f:
-      config.read_file(f)
-
-  return config
-
-def write_verfile():
-  if not args.newver:
-    return
-  util.write_verfile(args.newver, g_curver)
-
-def print_version_update(name, version):
-  if version is None:
-    task_dec()
-    return
-
-  oldver = g_oldver.get(name, None)
-  if not oldver or parse_version(oldver) < parse_version(version):
-    logger.info('%s updated version %s', name, version)
-    _updated(name, version)
-  else:
-    logger.info('%s current version %s', name, version)
-  task_dec()
-
-def _updated(name, version):
-  g_curver[name] = version
-
-  if args.notify:
-    msg = '%s updated to version %s' % (name, version)
-    notifications.append(msg)
-    notify.update('nvchecker', '\n'.join(notifications))
-
-def get_versions(config):
-  task_inc()
-  for name in config.sections():
-    task_inc()
-    get_version(name, config[name], print_version_update)
-  task_dec()
 
 def main():
   global args
 
   parser = argparse.ArgumentParser(description='New version checker for software')
-  parser.add_argument('files', metavar='FILE', nargs='*',
-                      help='software version source files')
   parser.add_argument('-n', '--notify', action='store_true', default=False,
                       help='show desktop notifications when a new version is available')
-  util.add_common_arguments(parser)
+  core.add_common_arguments(parser)
-
+  args = parser.parse_args()
-  args = util.parse_args(parser)
+  if core.process_common_arguments(args):
-  if util.process_common_arguments(args):
     return
 
-  if not args.files:
+  if not args.file:
     return
-
+  s = Source(args.file)
-  def run():
-    config = load_config(*args.files)
-    if args.oldver:
-      g_oldver.update(util.read_verfile(args.oldver))
-      g_curver.update(g_oldver)
-    get_versions(config)
 
   ioloop = IOLoop.instance()
-  ioloop.add_callback(run)
+  ioloop.add_callback(s.check)
   ioloop.start()
 
 if __name__ == '__main__':

nvchecker/source/regex.py

@@ -36,11 +36,12 @@ def get_version(name, conf, callback):
   ), **kwargs)
 
 def _got_version(name, regex, encoding, callback, res):
-  body = res.body.decode(encoding)
+  version = None
   try:
-    version = max(regex.findall(body), key=parse_version)
+    body = res.body.decode(encoding)
-  except ValueError:
+    try:
-    logger.error('%s: version string not found.', name)
+      version = max(regex.findall(body), key=parse_version)
-    callback(name, None)
+    except ValueError:
-  else:
+      logger.error('%s: version string not found.', name)
+  finally:
     callback(name, version)

nvchecker/tools.py

@@ -2,26 +2,31 @@
 
 import sys
 import argparse
+import logging
 
-from . import util
+from . import core
+
+logger = logging.getLogger(__name__)
 
 def take():
   parser = argparse.ArgumentParser(description='update version records of nvchecker')
+  core.add_common_arguments(parser)
   parser.add_argument('names', metavar='NAME', nargs='*',
                       help='software name to be updated')
-  util.add_common_arguments(parser)
+  args = parser.parse_args()
-
+  if core.process_common_arguments(args):
-  args = util.parse_args(parser)
-  if util.process_common_arguments(args):
     return
 
-  if not args.oldver or not args.newver:
+  s = core.Source(args.file)
-    sys.exit('You must specify old and new version records so that I can update.')
+  if not s.oldver or not s.newver:
+    logger.error(
+      "%s doesn't have both 'oldver' and 'newver' set, ignoring.", s
+    )
 
-  oldvers = util.read_verfile(args.oldver)
+  oldvers = core.read_verfile(s.oldver)
-  newvers = util.read_verfile(args.newver)
+  newvers = core.read_verfile(s.newver)
 
   for name in args.names:
     oldvers[name] = newvers[name]
 
-  util.write_verfile(args.oldver, oldvers)
+  core.write_verfile(s.oldver, oldvers)

nvchecker/util.py

@@ -1,85 +0,0 @@
-# vim: se sw=2:
-
-import os
-import sys
-import logging
-
-from .lib import nicelogger
-
-from . import __version__
-
-_DEFAULT_CONFIG = os.path.expanduser('~/.nvcheckerrc')
-
-def add_common_arguments(parser):
-  parser.add_argument('-i', '--oldver',
-                      help='read an existing version record file')
-  parser.add_argument('-o', '--newver',
-                      help='write a new version record file')
-  parser.add_argument('-c', metavar='CONFIG_FILE', default=_DEFAULT_CONFIG,
-                      help='specify the nvcheckerrc file to use')
-  parser.add_argument('-l', '--logging',
-                      choices=('debug', 'info', 'warning', 'error'), default='info',
-                      help='logging level (default: info)')
-  parser.add_argument('-V', '--version', action='store_true',
-                      help='show version and exit')
-
-def _get_rcargs():
-  args = sys.argv[1:]
-  args.reverse()
-  it = iter(args)
-
-  try:
-    f = next(it)
-    while True:
-      j = next(it)
-      if j == '-c':
-        break
-      f = j
-  except StopIteration:
-    if os.path.exists(_DEFAULT_CONFIG):
-      f = _DEFAULT_CONFIG
-    else:
-      return []
-
-  return [os.path.expandvars(os.path.expanduser(x))
-          for x in open(f, 'r').read().split()]
-
-def parse_args(parser):
-  args = _get_rcargs()
-  args += sys.argv[1:]
-  return parser.parse_args(args)
-
-def process_common_arguments(args):
-  '''return True if should stop'''
-  nicelogger.enable_pretty_logging(getattr(logging, args.logging.upper()))
-
-  if args.version:
-    progname = os.path.basename(sys.argv[0])
-    print('%s v%s' % (progname, __version__))
-    return True
-
-def safe_overwrite(fname, data, *, method='write', mode='w', encoding=None):
-  # FIXME: directory has no read perm
-  # FIXME: symlinks and hard links
-  tmpname = fname + '.tmp'
-  # if not using "with", write can fail without exception
-  with open(tmpname, mode, encoding=encoding) as f:
-    getattr(f, method)(data)
-  # if the above write failed (because disk is full etc), the old data should be kept
-  os.rename(tmpname, fname)
-
-def read_verfile(file):
-  v = {}
-  with open(file) as f:
-    for l in f:
-      name, ver = l.rstrip().split(None, 1)
-      v[name] = ver
-  return v
-
-def write_verfile(file, versions):
-  # sort using only alphanums, as done by the sort command, and needed by
-  # comm command
-  data = ['%s %s\n' % item
-          for item in sorted(versions.items(), key=lambda i: (''.join(filter(str.isalnum, i[0])), i[1]))]
-  safe_overwrite(file, data, method='writelines')
-

sources/arch_aur.ini

@@ -1,4 +1,7 @@
 # my AUR packages
+[__config__]
+oldver = ../records/arch_aur.txt
+newver = ../records/arch_aur.new.txt
 
 [cld2-svn]
 url = https://code.google.com/p/cld2/source/list

sources/arch_cn.ini

@@ -1,4 +1,7 @@
 # Arch Linux CN repository
+[__config__]
+oldver = ../records/arch_cn.txt
+newver = ../records/arch_cn.new.txt
 
 [aliedit]
 aur =

sources/arch_lilydjwg.ini

@@ -1,5 +1,8 @@
 # lilydjwg repository:
 # https://bbs.archlinuxcn.org/viewtopic.php?id=1695
+[__config__]
+oldver = ../records/arch_lilydjwg.txt
+newver = ../records/arch_lilydjwg.new.txt
 
 [3to2]
 aur

sources/sample_source.ini

@@ -1,3 +1,7 @@
+[__config__]
+oldver = old_ver.txt
+newver = new_ver.txt
+
 [fcitx]
 url = https://code.google.com/p/fcitx/
 regex = fcitx-([\d.]+)\.tar\.xz