Changeset 286

Show
Ignore:
Timestamp:
04/28/08 02:41:09 (7 months ago)
Author:
eric.dumin..@gmail.com
Message:

Unstable implementation.

Files:

Legend:

Unmodified
Added
Removed
Modified
Copied
Moved

  • branches/log_indexing_time_with_ferret/lib/picolena/templates/app/models/indexer.rb

    r283 r286  
     1require 'indexing_time' 
    12class Indexer 
    23  # This regexp defines which files should *not* be indexed. 
    …… 
    3839      indexing_list_chunks.each_with_thread{|chunk| 
    3940        chunk.each{|complete_path| 
    40           last_itime=index_time_dbm_file[complete_path] 
    41           if @from_scratch || !last_itime || File.mtime(complete_path)> Time._load(last_itime) then 
     41          if @from_scratch || File.mtime(complete_path)> IndexingTime::retrieve(complete_path) then 
    4242            add_or_update_file(complete_path) 
    4343          else 
    4444            log :debug => "Identical : #{complete_path}" 
    4545          end 
    46           index_time_dbm_file[complete_path] = Time.now._dump 
     46          IndexingTime::add(complete_path) 
    4747        } 
    4848      } 
    …… 
    7373    # ensures that a new Index is instantiated next time index is called. 
    7474    def close 
     75      IndexingTime::close_log 
    7576      @@index.close rescue nil 
    7677      # Ferret will SEGFAULT otherwise. 
    …… 
    8081     
    8182    # Checks for indexed files that are missing from filesytem 
    82     # and removes them from index & dbm file. 
     83    # and removes them from indexes. 
    8384    def prune_index 
    84       missing_files=index_time_dbm_file.reject{|filename,itime| File.exists?(filename) && Picolena::IndexedDirectories.any?{|dir,alias_path| filename.starts_with?(dir)}} 
    85       missing_files.each{|filename, itime| 
     85      IndexingTime::missing_documents.each{|filename, itime| 
    8686        index.writer.delete(:complete_path, filename) 
    87         index_time_dbm_file.delete(filename) 
     87        IndexingTime::log.delete(filename) 
    8888        log :debug => "Removed : #{filename}" 
    8989      } 
    9090      index.optimize 
     91      IndexingTime::log.optimize 
    9192    end 
    9293 
    …… 
    108109 
    109110    private 
    110      
    111     # Copied from Ferret book, By David Balmain 
    112     def index_time_dbm_file 
    113       @@dbm_file ||= DBM.open(File.join(Picolena::IndexSavePath, 'added_at')) 
    114     end 
    115  
    116111    def index_exists? 
    117112      index_filename and File.exists?(index_filename) 
    …… 
    157152      #  Ferret::Store::Lock::LockError 
    158153      index 
    159       # Opens dbm file to dump indexing time. 
    160       index_time_dbm_file 
     154      # Opens another index, to log IndexingTime of every document. 
     155      IndexingTime::log 
    161156      # ActiveSupport sometime raises 
    162157      #  Expected Object is NOT missing constant 

  • branches/log_indexing_time_with_ferret/lib/picolena/templates/config/environment.rb

    r280 r286  
    1 %w(rubygems paginator fileutils pathname logger thread dbm).each{|lib| require lib} 
     1%w(rubygems paginator fileutils pathname logger thread).each{|lib| require lib} 
    22 
    33# Uncomment below to force Rails into production mode when 

  • branches/log_indexing_time_with_ferret/lib/picolena/templates/lib/tasks/install_dependencies.rake

    r280 r286  
    3131    root_privileges_required! 
    3232    #TODO: Should load this list from defined PlainTextExtractor's 
    33     packages=%w{antiword poppler-utils odt2txt html2text catdoc unrtf mguesser libdbm-ruby1.8}.join(" ") 
     33    packages=%w{antiword poppler-utils odt2txt html2text catdoc unrtf mguesser}.join(" ") 
    3434    puts "Installing "<<packages 
    3535    system("apt-get install "<<packages)