How to do better logging? How to write out to a file?

I’ve searched here for this and only found one guy/thread talking about log4j and java.util.logging. I’ve use log4j in a corporate environment a lot, but I’m still getting used to the wpilib projects and how DriverStation and roboRio communicate. I’ve seen the logs that are automatically generated, and seen our System.out.println messages there, along with a bunch of other events and diagnostic messages. But I’d really like to produce a dedicated file with only our logging messages. Is this possible? I don’t understand how our program would write to a file on the PC. Is that through the Network Tables?

FYI, for some context, our auto routes behaving weird and erratically. We use different sets of PIDControllers for driving to a certain distance, and turning. We use the Command architecture. Sometimes the turns work, sometimes they don’t. For example, I’d really like to log things like PID output, or headings that we use in the command isFinished() to find out why one turn overshot and then continued to the next command.

I know there are also widgets in the SmartDashboard that can produce a graph, so we could see the values graphed there over time. Just haven’t had robot time yet to try this, and/or the problem isn’t reproducible enough in practice.

You can pretty easily log to a file on the roborio with Java’s builtin logging framework. Writing to a log file on the driver station would be more difficult.

What you can do, though, is to put the values you want to keep track of into NetworkTables and use Shuffleboard’s recording feature to save them. All sensors and actuators have telemetry enabled though NetworkTables already, so it’s easy to see everything you need in Shuffleboard

Sam, thanks for the response. If I could log to a file on the roborio, could I then transfer that file to the connected pc?

I’m thinking of another scenario we might use that for. We’re looking into Motion Profiling, and someone had the idea of recording a driver doing a route; recording the motor inputs from the joysticsk, so we could reproduce that route in auto. I assumed in that scenario that we would log the motor inputs, produce a file from that, which we would then later read back into an array as input to motion profile.

I’ll look into the Shuffleboard recording in the meantime. Thanks for the multiple options.

This is the exact thing that I’m currently working on. I have most of it figured out, my only issue now is it won’t write to the files that the roborio creates.

How it works now is by saving the data to a text file on a flashdrive that is plugged directly into the roborio. That way to get the data, all we have to do is plug the drive into a computer and copy the file over. The alternative to the flashdrive is to save it directly in the roborio, but to access files there you have to go to the web interface and download from there, which is a bit more complicated.

Anyway, in the code I create a FileWriter and then a PrintWriter/BufferedWriter. I’m trying to find the difference between the two. The file path for a flashdrive is “/U/folderForLogs/log.txt”. You have to create the folder in the flashdrive for it to work. Then from there, it should just be printWriter.println(data) or bufferedWriter.write(data), but neither will work. One thing I noticed is I didn’t have to create the file myself, because you can check and see if the file exists or not. From there you can have the roborio create it.

My only issue is it isn’t actually writing anything to the file. I don’t know much about this type of thing, but I think someone else might know why it isn’t saving. Maybe it doesn’t get the chance to actually save it? Here is my repository if you want to check it out.

I don’t recommend doing raw joystick input playback; it’s highly dependent on battery voltage and quality. Better to use closed-loop controls like the various path-planning libraries or PID controllers.

You need to call flush() for the text to get pushed to the file

Could you help me with setting up a SmartDashboard like that? I use iterative.

So shuffleboard is better than SmartDashboard? How would I make these NetworkTables and put them into Shuffleboard?

We use logback for our logging. You just add the logback JAR to your classpath, add a logback.xml file to your “src” directory, and then print log statements.

  1. Add the logback JAR to your classpath:
    If you use gradlerio (I strongly recommend it) then you just add this line to your build.gradle file in the “dependencies” section:

    compile group: 'ch.qos.logback', name: 'logback-classic', version: '1.2.3'

If not, you need to download the JAR here (click JAR link) and add it to the wpi source directory in your user directory (%HOMEPATH%\wpilib\user\java\lib). Note that you will need to add this JAR on each computer you deploy from (one of the reasons I recommend gradleRIO).

  1. Add logback.xml to the “src” directory:
    Here’s our logback.xml file.

<configuration>

	<timestamp key="TIME_BY_SECOND" datePattern="yyyy-MM-dd_HH.mm.ss"/>
	<timestamp key="TIME_BY_DAY" datePattern="yyyy-MM-dd"/>
	<timestamp key="TIME_BY_MONTH" datePattern="yyyy-MM"/>
	
	<property name="LOG_DATEFORMAT_PATTERN" value="HH:mm:ss.SSS"/>
	<property name="LOG_LEVEL_PATTERN" value="%5p"/>
	<property name="LOG_PATTERN" value="%d{${LOG_DATEFORMAT_PATTERN}} ${LOG_LEVEL_PATTERN} --- %-40.40logger{39} : %m%n"/>
	
	<property name="LOG_DIR_BASE" value="/home/lvuser/logs"/>
	<property name="LOG_DIR_DAILY" value="${LOG_DIR_BASE}/per-day"/>
	<property name="LOG_DIR_SESSION" value="${LOG_DIR_BASE}/per-session"/>
	
	<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
		<encoder>
			<pattern>${LOG_PATTERN}</pattern>
		</encoder>
	</appender>

	<appender name="FILE_BY_SECOND" class="ch.qos.logback.core.FileAppender">
		<file>${LOG_DIR_SESSION}/${TIME_BY_DAY}/session-log-${TIME_BY_SECOND}.log</file>
		<encoder>
			<pattern>${LOG_PATTERN}</pattern>
		</encoder>
	</appender>

	<appender name="FILE_BY_DAY" class="ch.qos.logback.core.FileAppender">
		<file>${LOG_DIR_DAILY}/${TIME_BY_MONTH}/daily-log-${TIME_BY_DAY}.log</file>
		<encoder>
			<pattern>${LOG_PATTERN}</pattern>
		</encoder>
	</appender>

	<root level="INFO">
		<appender-ref ref="CONSOLE"/>
		<appender-ref ref="FILE_BY_SECOND"/>
		<appender-ref ref="FILE_BY_DAY"/>
	</root>

</configuration>

  1. Print log statements. There are 5 logging levels: trace, debug, info, warn, and error. These control what will be printed so you can turn certain logging on/off. All levels above the current active level will be printed. For instance, if INFO is active, that means INFO, WARN and ERROR will print but TRACE and DEBUG will not. We have a special util to change the logging level via smartdashboard (we’ll be publishing it later this year) but you can just change the level in the logback.xml file ("") and redeploy.

public class YourRobot extends TimedRobot {
	private final Logger logger = LoggerFactory.getLogger(getClass());
	private TalonSRX elevatorMotor = new TalonSRX(0);

	@Override
	public void robotPeriodic() {
		logger.trace("This is a trace message!");
		logger.debug("This is a debug message!");
		logger.info("This is a info message!");
		logger.warn("This is a warn message!");
		logger.error("This is a error message!");
		
		// Instead of adding variable data using String concatenation:
		logger.info("Elevator position: " + elevatorMotor.getSelectedSensorPosition(0) + ". Elevator output: " + elevatorMotor.getMotorOutputVoltage() + " volts.");
		// prints: Elevator position: 1206. Elevator output: 5.67 volts.
		
		// You can add it using substitution. Any {} will be replaced by subsequent arguments
		logger.info("Elevator position: {}. Elevator output: {} volts.", elevatorMotor.getSelectedSensorPosition(0), elevatorMotor.getMotorOutputVoltage());
		// prints: Elevator position: 1206. Elevator output: 5.67 volts.

		// I personally like to add brackets around any variable data. Makes it easier to see what's dynamic in the message:
		logger.info("Elevator position: {}]. Elevator output: {}] volts.", elevatorMotor.getSelectedSensorPosition(0), elevatorMotor.getMotorOutputVoltage());
		// prints: Elevator position: [1206]. Elevator output: [5.67] volts.
	}
}

Then you can use WinSCP or the internet explorer admin console to get the log files from “/home/lvuser/logs”.

Let me know if you have any Q’s.

Also Sam might know a better way to include JARs on your classpath where they can be part of your project instead of in a user dir on each computer. That is the only way I knew how to do it using the standard ant build tool included.