Chief Delphi - File IO without overwriting file

Chief Delphi (http://www.chiefdelphi.com/forums/index.php)

- Java (http://www.chiefdelphi.com/forums/forumdisplay.php?f=184)

- - File IO without overwriting file (http://www.chiefdelphi.com/forums/showthread.php?t=124926)

blanketboyjo

19-01-2014 17:47

File IO without overwriting file

Hi, so I am trying to make the CRIO print out to a text file that I can read through ftp. I have it working to make the file, but everytime the code runs it overwrites the file, is there any way to make it so that new lines are added to the existing file?

Code:

PrintStream writer;

DataOutputStream myFile;

FileConnection con;



try {

con = (FileConnection)Connector.open("file:///test.txt", Connector.WRITE);

con.create();

myFile = con.openDataOutputStream();

writer = new PrintStream(myFile);

} catch (Exception e) {

}

writer.println("Message");

notmattlythgoe

20-01-2014 08:03

Re: File IO without overwriting file

Try

writer.append("Message");

blanketboyjo

21-01-2014 00:58

Re: File IO without overwriting file

I tried that but it says that append() is not a function of dataoutputstream. Is there anything else I can do?

Domenic Rodriguez

21-01-2014 11:44

Re: File IO without overwriting file

What if you tried something like this?

Code:

public void appendToFile() {

    FileConnection file;

    OutputStream out;

    PrintStream writer;



    try {

        // Open file for writing

        file = (FileConnection) Connector.open("file:///test.txt", Connector.WRITE);

        out = file.openOutputStream(Long.MAX_VALUE);

        writer = new PrintStream(out);



        // Append data to file

        writer.println("Message");



        // Close file connection

        writer.close();

        out.close();

        file.close();

    } catch (IOException ex) {

        // Handle exception

        ex.printStackTrace();

    }

}

The parameter passed to openOutputStream() specifies the offset position for writing to the file. Since the offset (should be) larger than the file size, it moves it to the end of the file, allowing you to append to the file instead of overwriting it.

Source: http://myossdevblog.blogspot.com/200...a-me-j2me.html

JohnGilb

25-01-2014 19:06

Re: File IO without overwriting file

I've been getting IO exceptions whenever I try anything along the lines of

OutputStreamWriter writer = new OutputStreamWriter(connection.openOutputStream(Lon g.MAX_VALUE));

or index out of bounds exceptions when I try:

writer.write(entry, (int)connection.fileSize(), entry.length))

(notably, for a file with about 15 characters in it, fileSize is returning 4831838208. That doesn't seem healthy or right.)

Has anybody been able to append to a file successfully on the robot?

NotInControl

25-01-2014 20:13

Re: File IO without overwriting file

Quote:

Originally Posted by blanketboyjo (Post 1329424)

Code:

PrintStream writer;

DataOutputStream myFile;

FileConnection con;



try {

con = (FileConnection)Connector.open("file:///test.txt", Connector.WRITE);

use Connector.Read_Write

try something like this

Code:

Connection conn = Connector.open(String fileName, Connector.READ_WRITE);

FileConnection fc = (FileConnection) conn;

if (!fc.exists()) {

fc.create();

}

OutputStream os = fc.openOutputStream(fc.fileSize());

After opening the file with the above code snippet if you are going to write something to the file then it will append at the end of the file content

Hope this helps,
Kevin

JohnGilb

27-01-2014 01:42

Re: File IO without overwriting file

Yeah, that's code that I would very much expect to work.

I am opening the fileconnection using

Code:

FileConnection c = (FileConnection) Connector.open(PATH, Connector.READ_WRITE);

and I have tried

Code:

OutputStreamWriter writer = new OutputStreamWriter(c.openOutputStream(c.fileSize()));

but that triggers an IO exception.

Have you made that code work on the cRIO? I have similar code that works on a PC, but something about the cRIO seems to be causing a problem...

NotInControl

27-01-2014 03:46

Re: File IO without overwriting file

Yes, what is the need to use the OutputStreamWriter class vs outputstream as in the example shown?

Do you know which IO exception is being thrown? Maybe we can track down the problem that way

simplistic approach:

Code:

try {

  ...

  throw new FileNotFoundException("This is an error message");

  ...

} catch (FileNotFoundException e) {

  System.out.println("File not found: " + e.getMessage());

  ...

} catch (EOFException e) {

  System.out.println("End of file reached: " + e.getMessage());

  ...

} catch (IOException e) { // catch all IOExceptions not handled by previous catch blocks

  System.out.println("General I/O exception: " + e.getMessage());

  e.printStackTrace();

  ...

}

joelg236

27-01-2014 11:45

Re: File IO without overwriting file

https://github.com/Team4334/atalibj/...TextFiles.java

I've tried dealing with this problem multiple times. Ended in just hacking a solution together.

If anyone's solutions end up working on a cRIO (and after reboots too!) - please let me know or submit a pull request. That'd be awesome.

Jon Stratis

27-01-2014 11:54

Re: File IO without overwriting file

What about creating a new log file each time? That's how things are handled with the server production systems I work with - each log file has a timestamp as part of the file name to help prevent overwrites.

I'm not sure how the clock works on the cRIO or what level of access we have towards getting a standardized timestamp, but there are other options as well. For example, you can have each log file generated with an incrementing numerical value (1.log, 2.log, etc). Keep the current value in a config file (log.config). When the program starts up, it reads the config file, increments by 1, writes the new value out to the config file (overwriting the old value) and creates the log file using that value.

This has the added benefit of giving you a separate log file for each run, which will help you keep everything organized.

NotInControl

27-01-2014 13:26

Re: File IO without overwriting file

I don't want to get off topic but,

Quote:

Originally Posted by Jon Stratis (Post 1333136)

I'm not sure how the clock works on the cRIO or what level of access we have towards getting a standardized timestamp, but... .

Unfortunately, the clock gets reset to Jan,1 1970 everytime the cRIO is started.

In the past I have played with reading system time from the Driverstation system clock and sending that over to the cRIO through the dashboard.

In Java there is no way to overwrite the system clock on the cRIO because the JRE doesn't allow access to the clock, you could do it with a JNI system call but I never went that far.

If you don't care about updating the system time on the crio, you could simply get a timestamp from the driverstation on robot startup and keep it in a variable and use that for time stamps, and use System time as a couter to count time since your "time sync".

Hope this helps,
Kevin

Joe Ross

27-01-2014 13:41

Re: File IO without overwriting file

Quote:

Originally Posted by NotInControl (Post 1333176)

Unfortunately, the clock gets reset to Jan,1 1970 everytime the cRIO is started.

The 8-slot cRIO has a battery backup for the RTC, but the 4-slot does not. See https://decibel.ni.com/content/message/33390#33390.

JohnGilb

28-01-2014 13:20

Re: File IO without overwriting file

If we can't get a generic append function working, we'll probably go with multiple incrementing log files, but there are some downsides:

-Since the robot can terminate unexpectedly, we have to flush our buffer fairly often to make sure we don't lose data due to a power failure or unexpected reboot
-Too many small files is a pain to read and process, so we would need to find a balance

I'll try to get more detail about our current IO exception (also, does anybody else have that problem where FileConnection.fileSize() returns an enormous number?), and if that doesn't work out, implement a solution like joelg236's. We already had some code that buffered about 7.9k worth of logs before flushing to disk, so we shouldn't be spending too much time on IO.

All times are GMT -5. The time now is 11:00.