Huge arrays - speed problems?

[Fabian]

Are there any speed issues in javascript with having really large
arrays? I know if the array gets large enough, bandwidth and download
time can be an issue, but does it take inordinate amounts of time to
access a really large array?


--
--
Fabian
Visit my website often and for long periods!

http://www.lajzar.co.uk

[Fabian]

Re: Huge arrays - speed problems?

Fabian hu kiteb:
[color=blue]
> Are there any speed issues in javascript with having really large
> arrays? I know if the array gets large enough, bandwidth and download
> time can be an issue, but does it take inordinate amounts of time to
> access a really large array?[/color]

Specifically, inordiante amounts of processor/disk access time.


--
--
Fabian
Visit my website often and for long periods!

http://www.lajzar.co.uk

[Lee]

Re: Huge arrays - speed problems?

Fabian said:[color=blue]
>
>Are there any speed issues in javascript with having really large
>arrays? I know if the array gets large enough, bandwidth and download
>time can be an issue, but does it take inordinate amounts of time to
>access a really large array?[/color]

That depends on what you consider to be "really large".

I wrote a simple script that creates an array of a given size,
loops through the array setting each element equal to 1,
<notes how long this has taken>
then loops through the array, adding up all those 1's, and
<notes how long this has taken>

I ran this in Netscape 7.1 and IE 6, with arrays of size
10,000, 100,000 and 1,000,000 elements, recording the time
in seconds to initialize the array, and then to access each
element (totalling them). All of the times were near zero
except at 1 million elements. That took Netscape 3 seconds
to initialize and 2 seconds to access, and IE 20 seconds(!)
to initialize but also 2 seconds to access all elements.

This was on a relatively slow processor.

[Fabian]

Re: Huge arrays - speed problems?

Lee hu kiteb:
[color=blue]
> Fabian said:[color=green]
>>
>> Are there any speed issues in javascript with having really large
>> arrays? I know if the array gets large enough, bandwidth and download
>> time can be an issue, but does it take inordinate amounts of time to
>> access a really large array?[/color]
>
> That depends on what you consider to be "really large".
>
> I wrote a simple script that creates an array of a given size,
> loops through the array setting each element equal to 1,
> <notes how long this has taken>
> then loops through the array, adding up all those 1's, and
> <notes how long this has taken>
>
> I ran this in Netscape 7.1 and IE 6, with arrays of size
> 10,000, 100,000 and 1,000,000 elements, recording the time
> in seconds to initialize the array, and then to access each
> element (totalling them). All of the times were near zero
> except at 1 million elements. That took Netscape 3 seconds
> to initialize and 2 seconds to access, and IE 20 seconds(!)
> to initialize but also 2 seconds to access all elements.[/color]

The maximum array size I was envisioning was going to be 10,000 by
20,000. I originally envisioned single character text, but there is no
functional reason they couldn't be nunbers, and numbers probably work
better in js anyway. Nice to know I am unlikely to hit any speed
barriers here.


--
--
Fabian
Visit my website often and for long periods!

http://www.lajzar.co.uk

[rf]

Re: Huge arrays - speed problems?


"Fabian" <lajzar@hotmail .com> wrote in message
news:bo7msf$1ac 74q$1@ID-174912.news.uni-berlin.de...[color=blue]
> Lee hu kiteb:
>[color=green]
> > Fabian said:[color=darkred]
> >>
> >> Are there any speed issues in javascript with having really large
> >> arrays? I know if the array gets large enough, bandwidth and download
> >> time can be an issue, but does it take inordinate amounts of time to
> >> access a really large array?[/color]
> >
> > That depends on what you consider to be "really large".
> >
> > I wrote a simple script that creates an array of a given size,
> > loops through the array setting each element equal to 1,
> > <notes how long this has taken>
> > then loops through the array, adding up all those 1's, and
> > <notes how long this has taken>
> >
> > I ran this in Netscape 7.1 and IE 6, with arrays of size
> > 10,000, 100,000 and 1,000,000 elements, recording the time
> > in seconds to initialize the array, and then to access each
> > element (totalling them). All of the times were near zero
> > except at 1 million elements. That took Netscape 3 seconds
> > to initialize and 2 seconds to access, and IE 20 seconds(!)
> > to initialize but also 2 seconds to access all elements.[/color]
>
> The maximum array size I was envisioning was going to be 10,000 by
> 20,000. I originally envisioned single character text, but there is no
> functional reason they couldn't be nunbers, and numbers probably work
> better in js anyway. Nice to know I am unlikely to hit any speed
> barriers here.[/color]

GAK. 10,000 by 20,000. Hmmm lets see, even if this were a character array
that would be <calculates /> a 190+ megabyte array. Any computer with less
than half a gig of memory would grind to a halt in the paging subsystem.

What *are* you trying to do?

Cheers
Richard.

[Fabian]

Re: Huge arrays - speed problems?

rf hu kiteb:
[color=blue]
> "Fabian" <lajzar@hotmail .com> wrote in message
> news:bo7msf$1ac 74q$1@ID-174912.news.uni-berlin.de...[color=green]
>> Lee hu kiteb:
>>[color=darkred]
>>> Fabian said:
>>>>
>>>> Are there any speed issues in javascript with having really large
>>>> arrays? I know if the array gets large enough, bandwidth and
>>>> download time can be an issue, but does it take inordinate amounts
>>>> of time to access a really large array?
>>>
>>> That depends on what you consider to be "really large".
>>>
>>> I wrote a simple script that creates an array of a given size,
>>> loops through the array setting each element equal to 1,
>>> <notes how long this has taken>
>>> then loops through the array, adding up all those 1's, and
>>> <notes how long this has taken>
>>>
>>> I ran this in Netscape 7.1 and IE 6, with arrays of size
>>> 10,000, 100,000 and 1,000,000 elements, recording the time
>>> in seconds to initialize the array, and then to access each
>>> element (totalling them). All of the times were near zero
>>> except at 1 million elements. That took Netscape 3 seconds
>>> to initialize and 2 seconds to access, and IE 20 seconds(!)
>>> to initialize but also 2 seconds to access all elements.[/color]
>>
>> The maximum array size I was envisioning was going to be 10,000 by
>> 20,000. I originally envisioned single character text, but there is
>> no functional reason they couldn't be nunbers, and numbers probably
>> work better in js anyway. Nice to know I am unlikely to hit any speed
>> barriers here.[/color]
>
> GAK. 10,000 by 20,000. Hmmm lets see, even if this were a character
> array that would be <calculates /> a 190+ megabyte array. Any
> computer with less than half a gig of memory would grind to a halt in
> the paging subsystem.
>
> What *are* you trying to do?[/color]

That was the maximum *theoretical* size. I would be somewhat surprised
if i ever actually used an array that size. Basically, I was thinking of
virtually dividing the world's surace into 3 mile squares, then creating
a pseudo-virtual reality.

Oh well.

Lets see. 100k^0.5 is 316. I guess I can do something with a grid that
size.


--
--
Fabian
Visit my website often and for long periods!

http://www.lajzar.co.uk

[Richard Cornford]

Re: Huge arrays - speed problems?

"rf" <making.it.up@t he.time> wrote in message
news:5uLpb.1782 63$bo1.32744@ne ws-server.bigpond. net.au...
<snip>[color=blue]
>GAK. 10,000 by 20,000. Hmmm lets see, even if this were a
>character array that would be <calculates /> a 190+ megabyte
>array. Any computer with less than half a gig of memory
>would grind to a halt in the paging subsystem.[/color]
<snip>

Even that seems an optimistic calculation as JavaScript has no character
data type, and if it did it would be a two byte Unicode character.

I would think that storing a single character would require JavaScript
to use a string value, probably implemented as a minimum of a 32 bit
unsigned integer describing the length plus the string date (2 bytes
minimum for a char), plus another 4 bytes so the Array could point to
the string.

And storing the character code as a number wouldn't help much either as
JavaScript would use a 64 bit double precision float for that.

Richard.

[Grant Wagner]

Re: Huge arrays - speed problems?

Richard Cornford wrote:
[color=blue]
> "rf" <making.it.up@t he.time> wrote in message
> news:5uLpb.1782 63$bo1.32744@ne ws-server.bigpond. net.au...
> <snip>[color=green]
> >GAK. 10,000 by 20,000. Hmmm lets see, even if this were a
> >character array that would be <calculates /> a 190+ megabyte
> >array. Any computer with less than half a gig of memory
> >would grind to a halt in the paging subsystem.[/color]
> <snip>
>
> Even that seems an optimistic calculation as JavaScript has no character
> data type, and if it did it would be a two byte Unicode character.
>
> I would think that storing a single character would require JavaScript
> to use a string value, probably implemented as a minimum of a 32 bit
> unsigned integer describing the length plus the string date (2 bytes
> minimum for a char), plus another 4 bytes so the Array could point to
> the string.
>
> And storing the character code as a number wouldn't help much either as
> JavaScript would use a 64 bit double precision float for that.
>
> Richard.[/color]

Not to mention that even to the most frugal representation to send the data
to the browser would consist of something like:

var a=[1,2,3,4,5...];

Even for a 10,000 member array consisting of a sequence of numbers (which of
course you wouldn't send to the browser anyway, since you could generate it
locally), you'd be looking at nearly 50,000 bytes of data sent to construct
the array:

9 x 1 digit numbers + 89 x 2 digit numbers + 899 x 3 digit numbers + 8999 x
4 digit numbers + 10000 x comma

If you start looking at sending an array of say, 10,000 5 digit product
codes, you're now looking at over 60K worth of data to download a browser
over an average 3K/s dialup line, so 20 seconds just to finish loading the
page.

Unless you have something really astonishing to sell or say, few people
would wait that long.

--
| Grant Wagner <gwagner@agrico reunited.com>

* Client-side Javascript and Netscape 4 DOM Reference available at:
*

http://devedge.netscape.com/library/manuals/2000/javascript/1.3/reference/frames.html

* Internet Explorer DOM Reference available at:
*

http://msdn.microsoft.com/workshop/author/dhtml/reference/dhtml_reference_entry.asp

* Netscape 6/7 DOM Reference available at:
* http://www.mozilla.org/docs/dom/domref/
* Tips for upgrading JavaScript for Netscape 7 / Mozilla
* http://www.mozilla.org/docs/web-deve...upgrade_2.html

[keyur shah]

Re: Huge arrays - speed problems?

Doesnt make much difference... Is your array in MBs or GBs :) wont be..
so no need to have any worry for large arrays... we have bandwidth these
days... dsl/cable/t1... why should one have speed issues.



Keyur Shah
Verizon Communications
732-423-0745

*** Sent via Developersdex http://www.developersdex.com ***
Don't just participate in USENET...get rewarded for it!

[HikksNotAtHome]

Re: Huge arrays - speed problems?

In article <3fa96c47$0$195 $75868355@news. frii.net>, keyur shah
<keyur@software klinic.com> writes:
[color=blue]
>Doesnt make much difference... Is your array in MBs or GBs :) wont be..
>so no need to have any worry for large arrays... we have bandwidth these
>days... dsl/cable/t1... why should one have speed issues.[/color]

Obviously, you are unaware (or choose to ingore) the fact that over half the
Internet still connects via dialup. So it *does* make filesize an issue.

http://www.jibbering.com/faq/#FAQ2_3

With regards to quoting when posting.
--
Randy

[Dr John Stockton]

Re: Huge arrays - speed problems?

JRS: In article <3fa96c47$0$195 $75868355@news. frii.net>, seen in
news:comp.lang. javascript, keyur shah <keyur@software klinic.com> posted
at Wed, 5 Nov 2003 21:31:51 :-[color=blue]
>Doesnt make much difference... Is your array in MBs or GBs :) wont be..
>so no need to have any worry for large arrays... we have bandwidth these
>days... dsl/cable/t1... why should one have speed issues.
>
>
>
>Keyur Shah
>Verizon Communications
>732-423-0745[/color]

That's rather a limited point of view; perhaps you are new to the
business? BTW, I don't think that number works from most telephones. A
communications firm should know about communication.

(A) Many users still have dial-up POTS connections, sometimes pay-by-
the-minute.

(B) Some users have portables, with low-bandwidth radio links.

(C) Since Javascript is a more-or-less general-purpose language, one can
perfectly use arrays whose contents have been generated by local
computation, and have not been transmitted in detail over the Internet.


Note : the OP was specifically referring to accessing large arrays.

--
© John Stockton, Surrey, UK. ?@merlyn.demon. co.uk Turnpike v4.00 IE 4 ©
<URL:http://jibbering.com/faq/> Jim Ley's FAQ for news:comp.lang. javascript
<URL:http://www.merlyn.demo n.co.uk/js-index.htm> JS maths, dates, sources.
<URL:http://www.merlyn.demo n.co.uk/> TP/BP/Delphi/JS/&c., FAQ topics, links.