Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upEachKey panics if more than 64 keys in path #56
Comments
|
Correction.. only supports 63 keys given signed int64 the way bitwiseFlags is populated. |
|
Yes it is, I have few ideas how to fix it without sacrificing performance. On Wednesday, 20 July 2016, Niek Sanders notifications@github.com wrote:
Sincerely yours Leonid Bugaev |
|
First off, thank you for writing and sharing this library! It's been incredibly helpful for a little project I'm working on. A wishlist item that relates to the many key lookup issue... I wish there was a way to iterate over a map and just have a callback invoked for each key/value pair:
For a many-key scenario like mine, I can just lookup the key in a Go map to see if it's interesting to me. Similarly, if you just want to chomp through all the keys in the map, this avoids having to determine/specify the keys you care about up front. |
|
I did a super-trashy hack implementation of my wishlist item above. Before I invoked EachKey ~3 times per json payload to pull my data. (63 keys per call). Now I use "WalkMap" once. Overall performance of my app, which includes many things unrelated to json, went from 20K msgs/sec to 30k msgs/sec. Before I was cpu-bound on json processing. Now I have free cpu and am bottlenecked elsewhere in my app. Like I mentioned, the code is hacked garbage hence no PR. But if you are curious it sits here: https://github.com/nieksand/jsonparser/blob/master/parser.go#L334 The actual change is here: |
|
I've been thinking of prototyping this feature, as well. If I get some free On Jul 21, 2016 01:08, "Niek Sanders" notifications@github.com wrote:
|
|
Very cool. I'm thinking it might be possible to rewrite EachKey to run on top of WalkMap. That would avoid the code duplication issue. |
daboyuka
mentioned this issue


I'm dealing with a large, flat json payload that has 100+ keys in a map.
I hit panics when using EachKey with the full list of json keys:
The problem code seems here:
https://github.com/buger/jsonparser/blob/master/parser.go#L236
If I'm reading it right, there is a limit of 64 keys per EachKey lookup before the int64 bitmask overflows.
Limit should probably be documented and code should return an error rather than panic.