|
|
|
|
|
by JesseAldridge
3632 days ago
|
|
I think Sublime Text should be able to handle fairly large json files without a problem. If not, I think I would just split data into multiple smaller files. Something like the python code below should work, assuming the file can fit in memory. If not, I assume you can find some json lib that can work in streaming mode and then do the same thing. import json, os
json_input = '''{
"foo": 1,
"bar": 2,
"baz": 3,
"bug": 4,
"thing": 5
}'''
entries_per_group = 2
if not os.path.exists('sub_files'):
os.mkdir('sub_files')
main_d = json.loads(json_input)
iter = main_d.iteritems()
for group_count in range(10 ** 6):
sub_d = {}
try:
for _ in range(entries_per_group):
k, v = iter.next()
sub_d[k] = v
except StopIteration:
break
finally:
json_output = json.dumps(sub_d, indent=2)
with open(os.path.join('sub_files', '{}.json'.format(group_count)), 'w') as f:
f.write(json_output)
|
|