wiki.ipfire.org

The community-maintained documentation platform of IPFire

User Tools

Site Tools


devel:proxy:update-accelerator

Plans for next version of Update-Accelerator 3.0

This page is for sharing ideas of new functions and improvements of the Squid-Addon “Update Accelerator”.

Current situation:

  • The current Version have some missing features and use for example not the best way of storing metadata.
  • Adding sources and maintaining existing sources is only possible over console / SSH, nor easy to backup and is not update-safe while an system-upgrade.
  • While a filecheck run / calling statistics- or maintain-page in webui, there is a huge IO-Load on the system, caused by the way of storing the metadata of each updatefile (filebased)
  • Adding / Changeing new Vendors and or their grafics is only possible over console / SSH
  • No precheck if a RegEx is correct for a new source, which cause to stop Squid at all.
  • No precheck if a RegEx is already defined
  • Updates are always downloaded twice, when new
  • Updates, which masquerades special chars with f.e. %20 are not correctly stored in the update cache and caused to always reload the updates
  • No option to select the way to check when a file is “old”, as example to select filesize and/ or filetime

Any Requirements / Ideas for new version:

  • ICAP-based interface to Squid instead of current redirector
  • Storing metadata of update files in a file-DB(sqllite) or via DB-Interface (Postgress / MySQL ?)
  • Storing vendors / sources in a file-DB(sqllite) or via DB-Interface
  • Extending the web-ui to add the possibility to add new vendors / sources / grafics
  • Including prechecks of entered RegEx if they are working in general and matching some example urls
  • Including prechecks if RegEx is allready defined
  • Including an assistant to build RegEx ?
  • Double load isn't avoidable. The requester cannot wait until the caching download has finished.
  • Sorting the file-list by all Metadata (Status/Vendor/Filename/Filesize/LastAccess)
  • Adaptive sorting of the vendor list based on usage. If you test the most used sources first you an improvement in speed.
  • Compiling of all source URI's and RE's to one RE. This speeds up scanning. This can be done by Perl module outside update accelerator similiar to languages update.
  • Definitions can be stored in files named <source>.<type> in definition directory. Compilation does the conversion axeldef/*.* –> %search

(I think, we should go away from storing definitions or metadata on a file-based way, exept backup or import them. - nightshift

I thought of the definitions to import of course - bebima )

  • Categories should be named “sources” and not “vendors”, if we step to a more general file cache

(Good point - nightshift)

Basic requirements

Software / Version

  • Python / ??
  • Squid / ??
  • nginx / ?? ( is this not oversized? )
  • Perl / >5.10
  • Perl module Regex::Assemble for compiling URI's of a “vendor”

Please extend the above list, if you have a requirement or idea for the new version wich is not listed here.

devel/proxy/update-accelerator.txt · Last modified: 2013/03/07 18:22 by bebima