* Language detection with CLD3 POC * Update spec/models/article_spec.rb Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update spec/models/article_spec.rb Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update spec/models/article_spec.rb Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update spec/models/article_spec.rb Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update spec/models/article_spec.rb Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update spec/models/article_spec.rb Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update app/models/article.rb Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Clean up tests * Clean up tests * Move language detection to service * rubocop * Update app/services/languages/detection.rb Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * More linting cleanup * Remove allow_any_instance_of * Update tests to better exercise different paths * Fix flaky spec * Update spec/services/languages/detection_spec.rb Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update spec/models/article_spec.rb Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update spec/models/article_spec.rb Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update spec/models/article_spec.rb Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update spec/models/article_spec.rb Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update spec/services/languages/detection_spec.rb Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update spec/services/languages/detection_spec.rb Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update spec/services/languages/detection_spec.rb Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update spec/services/languages/detection_spec.rb Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
22 lines
474 B
Ruby
22 lines
474 B
Ruby
module Languages
|
|
class Detection
|
|
attr_reader :text
|
|
|
|
PROBABILITY_THRESHOLD = 0.5
|
|
|
|
def self.call(...)
|
|
new(...).call
|
|
end
|
|
|
|
def initialize(text)
|
|
@text = text
|
|
end
|
|
|
|
def call(identifier: CLD3::NNetLanguageIdentifier.new(0, 1000))
|
|
language_outcome = identifier.find_language(text)
|
|
return unless language_outcome.probability > PROBABILITY_THRESHOLD && language_outcome.reliable?
|
|
|
|
language_outcome.language
|
|
end
|
|
end
|
|
end
|