Design problem: dealing with multiple time frames.

[arnaudk]

I am trying to come up with a class design to deal with asynchronous data to be stored and analyzed over multiple time frames and could really use some design input. This is a rather long question but seeing the slowdown in the number of postings, I'm hoping some of you will have more time. (And this is not some kind of coursework question!)

I obtain data elements from an asynchronous data source (i.e. the data arrives at irregular time intervals) and wish to place them in a container of fixed length in time, which I call a "DataChart" . This will be implemented using a deque. As a new data element arrives from the source, I remove all elements that are, say, 10mins older than the new data element from the back of the deque before pushing the new element on the front. This "10min DataChart" is my shortest time frame.

When the deque fills (i.e. the oldest DataElement is 10mins old and is about to be discarded as a new element arrives), I take a snapshot of the deque by averaging over its contents and store this average in another container of fixed width in time, a "1 day AveragesChart" which is implemented as a deque of 10min-AverageElements . Every 10mins the contents of the shortest timeframe, 10min-DataChart, are renewed and thus a new 10min-average element is generated to be stored in the 1-day-chart AveragesChart.

Likewise, I have a 1 month chart containing "1-day averages" made by averaging over the 1-day chart of 10min-averages when its contents are renewed once a day, and so on.

So far, I have a base class Chart which is the interface, and two derived classes DataChart and AveragesChart, each of which contain a deque and methods to add new data, etc.

Then I have a class ChartSet which holds all the charts, namely, the 10-min DataChart, the 1-day AveragesChart, the 1-month AveragesChart, etc, in its vector of Chart base handles.

In this way, I can have any number of time frames spanning arbitrary lengths in time, the shortest will always be a DataChart comprised of DataElements, and subsequent time frames AveragesCharts comprised of AverageElements .

So here are my questions,

1. I have a pure virtual method addelement() in the base class, Chart, to update a given chart with new data. The overloaded addelement() in the derived DataChart takes DataElements. By contrast, addelement() in the derived AveragesChart takes AverageElements . This is my first dilema.
   I could make a base class "Element" to shoehorn derived DataElement and AverageElement objects into the base class addelement() method which can now just take generic Elements. But a DataElement and AverageElement objects are fundamentally different so I'd like to avoid this, plus I'd have to rewrite other portions related to the elements themselves, etc. It would be great if I could overload the base class pure virtual function like addelement(void ) in a derived class as addelement(Data Element& data) but then the compiler complains that I haven't supplied an implementation of addelement(void ) in derived classes. Your thoughts on this?
2. Later, I'd like to compute other quantities from the charts, like a discrete Fourier transform, for example. I can add methods begin() and end() to the chart interface which return iterators to the Deques they contain and use those iterators in computations. Do you think this is a good idea? Does it expose my implementation of the Charts?
   Another way would be to have something like a visitor, or pass function objects to the charts containing the algorithm to be computed on the elements. I'm not sure what the design of that should be yet. Is that a good, extensible design?
3. Which class do you think should have the responsibility of generating anAverageElemen t for the longer timeframe Chart when a shorter time frame Chart fills, the ChartSet object which encapsulates the Charts, or the Charts themselves?

Any other comments would be greatly appreciated.

[arnaudk]

Is the question too long-winded?

[Laharl]

I can answer a couple small parts of this, though the whole of it...not a clue. I also think the ChartSet class should generate the new charts rather than the Charts themselves.

You can always define addelement(void ) with
[code=cpp]addelement(void ){}[/code]. I'm not sure how well that would work for you, though.

[RedSon]

Its not to long-winded, it is too technical. You discuss many implementation details that are irrelevant to the problem. What you need to do is explain your problem domain and then explain from a high level what your designed solution is (skipping the implementation details), then you might get more responses.

I think you are trying to have multiple queues then for each queue you average the data in one queue and then stick that value into another queue?

It seems like it might be better to just have one larger queue say data collected over 30 days then average for different time values.

[arnaudk]

I also think the ChartSet class should generate the new charts rather than the Charts themselves.

Yes, I agree that does make more sense.

You can always define addelement(void ) with
[code=cpp]addelement(void ){}[/code].

Sure, that will get around the problem but it kind of defeats the purpose of having an abstract base class as an interface; if the interface contains an addelement() method, that should force derived classes to have a meaningful addelement(Elem ent&) method. By adding an addelement(void ) method in derived classes I effectively circumvent this requirement.

I'm starting to think my class model is incorrect: I have two unrelated types of elements which I want to hold in two related types of chart, one for each element type. This seems contradictory - for consistency, it seems that either

1. the charts should be unrelated through inheritance, or
2. the elements should be related through inheritance.

The problem with (a) is that I'd like to specify a base class interface for the charts which necessarily relates charts through inheritance, and the problem with (b) is that the elements are radically different and have very different methods and members so a base class for those would be kind of a hack. That's one of my dilemmas.

I think you are trying to have multiple queues then for each queue you average the data in one queue and then stick that value into another queue?

Yes, that's one aspect of what I'm trying to do.

It seems like it might be better to just have one larger queue say data collected over 30 days then average for different time values.

That was my initial idea. But it would be prohibitive insofar as memory is concerned because of the prodigious amount of data I have (spanning several years). In fact, I am concerned with the impact of the past on the present and future, so I only need a vague average of data in the far distant past since the precise details will have very little impact on what is happening in the present compared to the recent past (where I have shorter averages) or the very recent past (where I don't examine averages but the actual data elements instead). This fact saves me a lot of memory.