FRC code search: a great project for someone to start

[euhlmann]

Originally Posted by Jaci

4) Just pass in your query as a git grep.

Or use an actual database engine's full text search, which should be far more efficient.


Btw, when I tried searching for usages of WPILib classes with my mongo/node implementation, I found more than a few top results being the source for that WPILib class, since it seems many teams like to keep entire copies of WPILib in their repos. I wonder if there's a way to filter those out.

[virtuald]

Originally Posted by euhlmann

Or use an actual database engine's full text search, which should be far more efficient.


Btw, when I tried searching for usages of WPILib classes with my mongo/node implementation, I found more than a few top results being the source for that WPILib class, since it seems many teams like to keep entire copies of WPILib in their repos. I wonder if there's a way to filter those out.

You could probably hash every file, and store them indexed by hash. Then each teams code is just a list of paths mapped to hashes. When you do a search, it could show each of the places where the file is used.

If you wanted to be super fancy, you could split by subroutine, and hash those (excluding whitespace).

[Jaci]

Originally Posted by euhlmann

Or use an actual database engine's full text search, which should be far more efficient.


Btw, when I tried searching for usages of WPILib classes with my mongo/node implementation, I found more than a few top results being the source for that WPILib class, since it seems many teams like to keep entire copies of WPILib in their repos. I wonder if there's a way to filter those out.

Git grep is actually quite fast, and way more space efficient than most database engines. Most database engines don't have paging sizes large enough for an entire source file, and so you're left with subpar lookup speeds as compared to git grep that can directly look through the changes to files. Obviously this needs some testing, but this is the theory behind it anyway.

Git grep also has the advantage that you can call it with `git rev-list --all` to search for occurrences of a string over all commits, ever. Likewise, you can grep subdirectories, discounting any directories that resemble that of the wpilib source.