Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse the data from JSON file without loading the whole JSON file #97

Open
machineCYC opened this issue Aug 23, 2021 · 0 comments
Open

Comments

@machineCYC
Copy link

Since I have a big JSON file(20GB), I want to parse the data from that JSON file, but I do not want to load the whole JSON file

So, I only have the JSON file path, and I want to get the specific data in JSON file

such as below is a JSON file, I want to parse the "ColC" data

data = {
    "ColA":"XXX",
    "ColA":"XXX",
    "ColB":"XXX",
    "ColC":[
        {"ColC1":"ZZZ1", "time":"TTT1"},
        {"ColC2":"ZZZ2", "time":"TTT2"},
        {"ColC3":"ZZZ3", "time":"TTT3"},
        {"ColC4":"ZZZ4", "time":"TTT4"},
        ...
        {"ColC100000":"ZZZ100000", "time":"TTT100000"},
    ],
    "ColD":[
        {"ColD1":"ZZZ1", "time":"TTT1"},
        {"ColD2":"ZZZ2", "time":"TTT2"},
        {"ColD3":"ZZZ3", "time":"TTT3"},
        {"ColD4":"ZZZ4", "time":"TTT4"},
        ...
        {"ColD100000":"ZZZ100000", "time":"TTT100000"},
    ],
    "ColE":"XXX",
    "ColF":"XXX",
    ....
    "ColZ":"XXX",
}

I am not sure python-jsonpath-rw can do this or not,

have any examples can share with me?

@machineCYC machineCYC changed the title python-jsonpath-rw can parse the data from JSON file without loading the whole JSON file Parse the data from JSON file without loading the whole JSON file Aug 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant