Changeset 323

Show
Ignore:
Timestamp:
05/05/08 03:22:33 (7 months ago)
Author:
eric.dumin..@gmail.com
Message:

Moved log methods to IndexerLogger?

Files:

Legend:

Unmodified
Added
Removed
Modified
Copied
Moved

  • trunk/Manifest.txt

    r301 r323  
    5151lib/picolena/templates/lang/ui/fr.yml 
    5252lib/picolena/templates/lib/core_exts.rb 
     53lib/picolena/templates/lib/indexer_logger.rb 
    5354lib/picolena/templates/lib/plain_text_extractor_DSL.rb 
    5455lib/picolena/templates/lib/plain_text_extractors/adobe.pdf.rb 

  • trunk/lib/picolena/templates/app/models/indexer.rb

    r322 r323  
    33# one IndexWriter exists at any given time, even when used in a multi-threaded 
    44# way. 
     5require 'indexer_logger' 
    56class Indexer 
    67  # This regexp defines which files should *not* be indexed. 
    …… 
    2324      # Forces Finder.searcher and Finder.index to be reloaded, by removing them from the cache. 
    2425      Finder.reload! 
    25       log :debug => "Indexing every directory" 
    26       start=Time.now 
     26      logger.start_indexing 
    2727      Picolena::IndexedDirectories.each{|dir, alias_dir| 
    2828        index_directory_with_multithreads(dir) 
    2929      } 
    30       log :debug => "Now optimizing index" 
     30      logger.debug "Now optimizing index" 
    3131      index.optimize 
    3232      index_time_dbm_file['last']=Time.now._dump 
    3333      @@do_not_disturb_while_indexing=false 
    34       log :debug => "Indexing done in #{Time.now-start} s." 
     34      logger.show_report 
    3535    end 
    3636 
    …… 
    3939    # @@threads_number chunks, and create a new indexing thread for every chunk. 
    4040    def index_directory_with_multithreads(dir) 
    41       log :debug => "Indexing #{dir}, #{@@threads_number} threads" 
     41      logger.debug "Indexing #{dir}, #{@@threads_number} threads" 
    4242 
    4343      indexing_list=Dir[File.join(dir,"**/*")].select{|filename| 
    …… 
    5454            add_or_update_file(complete_path) 
    5555          else 
    56             log :debug => "Identical : #{complete_path}" 
     56            logger.debug "Identical : #{complete_path}" 
    5757          end 
    5858          index_time_dbm_file[complete_path] = Time.now._dump 
    …… 
    7373        document.merge! PlainTextExtractor.extract_content_and_language_from(complete_path) 
    7474        raise "empty document #{complete_path}" if document[:content].strip.empty? 
    75         log :debug => ["Added : #{complete_path}",document[:language] && " ("<<document[:language]<<")"].join 
     75        logger.add_document document 
    7676      rescue => e 
    77         log :debug => "\tindexing without content: #{e.message}" 
     77        logger.reject_document document, e 
    7878      end 
    7979      index << document 
    …… 
    101101        index.writer.delete(:complete_path, filename) 
    102102        index_time_dbm_file.delete(filename) 
    103         log :debug => "Removed : #{filename}" 
     103        logger.debug "Removed : #{filename}" 
    104104      } 
    105105      index.optimize 
    …… 
    138138 
    139139    private 
     140 
     141    def logger 
     142      @@logger ||= IndexerLogger.new 
     143    end 
    140144     
    141145    # Copied from Ferret book, By David Balmain 
    …… 
    150154    def index_filename 
    151155      Dir.glob(File.join(Picolena::IndexSavePath,'*.cfs')).first 
    152     end 
    153  
    154     def log(hash) 
    155       hash.each{|level,message| 
    156         IndexerLogger.send(level,message) 
    157       } 
    158156    end 
    159157 

  • trunk/lib/picolena/templates/config/environment.rb

    r280 r323  
    88RAILS_GEM_VERSION = '2.0.2' unless defined? RAILS_GEM_VERSION 
    99 
    10 IndexerLogger=Logger.new($stdout) 
    11  
    1210# Bootstrap the Rails environment, frameworks, and default configuration 
    1311require File.join(File.dirname(__FILE__), 'boot') 

  • trunk/lib/picolena/templates/config/environments/development.rb

    r148 r323  
    1717# Don't care if the mailer can't send 
    1818config.action_mailer.raise_delivery_errors = false 
    19  
    20  
    21 IndexerLogger.level = Logger::DEBUG 

  • trunk/lib/picolena/templates/config/environments/production.rb

    r148 r323  
    1818# Disable delivery errors, bad email addresses will be ignored 
    1919# config.action_mailer.raise_delivery_errors = false 
    20  
    21 IndexerLogger.level = Logger::INFO 

  • trunk/lib/picolena/templates/config/environments/test.rb

    r148 r323  
    2121# ActionMailer::Base.deliveries array. 
    2222config.action_mailer.delivery_method = :test 
    23  
    24  
    25 IndexerLogger.level = Logger::WARN 

  • trunk/lib/picolena/templates/lib/core_exts.rb

    r322 r323  
    4141    # and removes added nils. 
    4242    (self+[nil]*i).enum_slice(n).to_a.transpose.collect{|e| e.compact} 
     43  end 
     44end 
     45 
     46class Hash 
     47  def add(category) 
     48    self[category]||={:size=>0} 
     49    self[category][:size]+=1 
    4350  end 
    4451end 

  • trunk/lib/picolena/templates/lib/plain_text_extractors/plain_text.rb

    r320 r323  
    44  aka "plain text file" 
    55  with {|source| 
    6     raise "binary file #{source}" unless File.plain_text?(source) 
     6    raise "binary file" unless File.plain_text?(source) 
    77    encoding=File.encoding(source) 
    88    if encoding.empty? then 

  • trunk/lib/picolena/templates/spec/models/plain_text_extractor_spec.rb

    r320 r323  
    3131  it "should not extract content of binary files" do 
    3232    bin_file="spec/test_dirs/indexed/others/BIN_FILE_WITHOUT_EXTENSION" 
    33     lambda{PlainTextExtractor.extract_content_from(bin_file)}.should raise_error(RuntimeError, "binary file "<<bin_file) 
     33    lambda{PlainTextExtractor.extract_content_from(bin_file)}.should raise_error(RuntimeError, "binary file") 
    3434  end 
    3535end