Sequence of characters problem

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • arturmuller
    New Member
    • Feb 2020
    • 1

    #1

    Sequence of characters problem

    I have a sequence of characters. The characters are ordered in chronological order. I am looking for an algorithm to group the sequence and remove errors in the data. I am having an hard time to explain in words/maths what the requirement of fixing the sequence is but something like "the outcome is to group as many characters as possible in a chain of constant letters" and minimum sequence length should be a setting. Below I am trying to visualize by some examples what I mean (3 chars minimum):

    AAACCAACA => AAAAAAAAA

    ACACAAAAA => AAAAAAAAA

    AYYYYYYYA => YYYYYYYYY

    YYYYAAYYY => YYYYYYYYY

    AYAYAYYYY => YYYYYYYYY

    And for longer sequences it becomes a little more tricky

    AYAYAYAYAAAAAAA AAAYAYYYYYYYYYY YYYY => AAAAAAAAAAAAAAA AAAYYYYYYYYYYYY YYYY

    HTHTTHHHTTHHH => TTTTTHHHHHHHH

    TTOAOAOOAATTA => OOOOOOOOAAAAA

    This is bit tricky because even thought the 3 O's aren't directly next to each other, they are still the most probable uninterrupted sequence of characters.

    Does anyone know of any algorithm (Machine learning?) or similar (term of this problem, problem name) that can do this type of error-code correction?

    Thank you in advance
  • Rabbit
    Recognized Expert MVP
    • Jan 2007
    • 12517

    #2
    If you want to write a rules based engine, then you'll need to come up with more precise rules. For example, you will need to define why AYAYAYYYY maps to YYYYYYYYY and not AAAAAYYYY.

    If you want to write a fuzzy matching algorithm, then there's plenty to choose from, n-gram matching and levenshtein distance comes to mind.

    If you want to use machine learning, as in a neural network, then you will need tens of thousands, if not more, of manually verified examples to feed the network for training.

    Or, don't fix the data, instead, fix the source of the error. Go back to the system before you receive the data and fix the issue that is sending you incorrect data.
    Last edited by Rabbit; Feb 12 '20, 10:41 PM.

    Comment

    Working...