## Java - writing int arrays

In this forum members can discuss topics about specific programming languages.
PurpleBlu3s
Posts: 73
Joined: Mon Sep 19, 2011 5:49 pm

### Java - writing int arrays

Hi, I'm hoping to write two fairly large integer arrays to a file (about 1000000-2000000 ints per array) to a file, and read them back again. I don't have much file IO experience in general other than reading simple text files, so I would appreciate it if anyone can help me with this.

Thanks.

EDIT: I'll just add that I've tried a naive approach of building a string of the array, but it takes longer than I can be bothered to wait to build the string with 1000000 ints, unsurprisingly.

Argaen
Posts: 10
Joined: Fri Apr 10, 2009 10:19 pm

### Re: Java - writing int arrays

Assuming you already have populated the arrays, you can write them to a File with a code like this:

Code: Select all

void writeToFile(int[] array, String fileName) throws IOException{
FileWriter fw = new FileWriter(new File(fileName));
for(int i in array){
fw.write(i + " ");
}
fw.close(); // Really important!
}
That would write every number in a single line, separated by a single space between them. The write() method is like System.out.print(), it does not print a line feed.
To append something to the end of the file, without losing what was already there you can call the FileWriter constructor with a second parameter like this:

Code: Select all

FileWriter fw = new FileWriter(new File(fileName), true);
The official documentation of the FileWriter class is here: http://download.oracle.com/javase/6/doc ... riter.html.

To read them I would recomend the Scanner class, with something like this:

Code: Select all

ArrayList<Integer> readFile(String fileName) throws IOException{
Scanner s = new Scanner(new File(fileName));
ArrayList<Integer> list = new ArrayList<Integer>();
while(s.hasNextInt()){
}
return list;
}
Est Sularis Oth Mithas.

PurpleBlu3s
Posts: 73
Joined: Mon Sep 19, 2011 5:49 pm

### Re: Java - writing int arrays

Thanks. I was essentially doing what you did in your writeToFile method, but I was using BufferedWriter and I was building the string and then writing the string. I don't understand why what I did before would be so much slower than this - it took about 1 second to write a 1.6 million length array to a file this way, but it took longer than I could be bothered to wait the old way (>20 minutes).

thundre
Posts: 356
Joined: Sun Mar 27, 2011 9:01 am

### Re: Java - writing int arrays

PurpleBlu3s wrote:Thanks. I was essentially doing what you did in your writeToFile method, but I was using BufferedWriter and I was building the string and then writing the string. I don't understand why what I did before would be so much slower than this - it took about 1 second to write a 1.6 million length array to a file this way, but it took longer than I could be bothered to wait the old way (>20 minutes).
Building a String with a StringBuilder, or with String concatenation in a loop? String construction by repeated concatenation in Java is notoriously slow, which is why StringBuffer and StringBuilder were added to the API.

hk
Posts: 10488
Joined: Sun Mar 26, 2006 9:34 am
Location: Haren, Netherlands

### Re: Java - writing int arrays

I'm no Java programmer, but converting integers to string to write them to a file did not seem a very good idea to me and I wondered if there was a way to write them as integers.
Some Googling brought me to this:
http://www.java-examples.com/write-int- ... tputstream
and its companion:

TripleM
Posts: 382
Joined: Fri Sep 12, 2008 2:31 am

### Re: Java - writing int arrays

- ignore this -

A DataOutputStream won't actually make the file you are writing to readable as text if that is important; they'd just be written as bytes. It is also about 50 times slower (based on a quick test) than a plain FileWriter.

You could always wrap the FileWriter in a PrintWriter, which provides methods like print(int x) - though all that does is convert x to a String and call write(), since a file like that is, after all, just one long string.

The method mentioned above is the best to use, really.
Last edited by TripleM on Wed Nov 16, 2011 9:32 am, edited 2 times in total.

hk
Posts: 10488
Joined: Sun Mar 26, 2006 9:34 am
Location: Haren, Netherlands

### Re: Java - writing int arrays

Odd language, that Java.
I would say that writing 4 bytes would be much faster than the equivalent string that might have 9 bytes.
Not to speak about converting to string and back again.
So there must be something wrong with Java's implementation.

TripleM
Posts: 382
Joined: Fri Sep 12, 2008 2:31 am

### Re: Java - writing int arrays

Actually sorry, ignore the figure above, it appears the FileWriter was buffered while the DataOutputStream wasn't. If I buffer the latter, it is of course faster.

So just depends on whether you want the file readable as text or not.

hk
Posts: 10488
Joined: Sun Mar 26, 2006 9:34 am
Location: Haren, Netherlands

### Re: Java - writing int arrays

Just another thing:
Before writing the integers to a file it might be useful to inspect the data.
I encountered this when I wanted to write a large number of consecutive primes to a file, to read them in again when necessary.
As it turned out no prime gap in the range exceeded 510. So I divided every gap by 2 and stored only the prime gaps divided by 2 as bytes (buffered reading and writing of course).
The file then contained the following numbers as bytes: 2,3,1,1,2,1,2... meaning 2,3,3+2*i=5,5+2*1=7,7+2*2=11,11+2*1=13,13+2*2=17,....
It turned out to be the fastest way to save and retrieve enough information about those primes to recalculate them when reading them in again.
It turned out too, that recalculating them using an efficient prime sieve was even faster. So I haven't ever used that file after its creation. It's still somewhere on my computer.

PurpleBlu3s
Posts: 73
Joined: Mon Sep 19, 2011 5:49 pm

### Re: Java - writing int arrays

Thanks for the replies. Yes I was using concatenation in a loop. It's all irrelevant now because the reason I was doing it was to make it easier to try out different colouring methods on Nebulabrots, but it turns out not to work as I only realised after coding it. If anyone happens to know much about Nebulabrot and if it's feasible to save the data that describes it so that different colouring methods can be used without having to redraw the whole thing, then I would love to know. Nebulabrot are Buddhabrot but with different colouring. Buddhabrot are an interesting alternative way of drawing the Mandelbrot set. Here is an example of a Nebulabrot my program has produced (2000x2000 pixels; took about 16 hours to produce - this could definitely be a lot faster, but it's just the best way I've been able to code):
http://dl.dropbox.com/u/16319566/8000it ... ts-01c.png

thundre
Posts: 356
Joined: Sun Mar 27, 2011 9:01 am

### Re: Java - writing int arrays

hk wrote:It turned out to be the fastest way to save and retrieve enough information about those primes to recalculate them when reading them in again.
It turned out too, that recalculating them using an efficient prime sieve was even faster. So I haven't ever used that file after its creation. It's still somewhere on my computer.
I had a similar experience with XML data. Saving a very large XML stream, I wondered if compressing the data using ZipOutputStream was feasible. Testing showed that not only was it feasible, it was faster, because the process was I/O bound, and the compressed data required less I/O.

Java has a good RandomAccessFile class. If you wanted to store a prime list that actually exceeds your available memory, you could use that.