Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Profiler only generates xplane.pb (Tensorflow 2.11 & 2.12) #613

Open
HLneoh opened this issue Apr 28, 2023 · 3 comments
Open

Profiler only generates xplane.pb (Tensorflow 2.11 & 2.12) #613

HLneoh opened this issue Apr 28, 2023 · 3 comments

Comments

@HLneoh
Copy link

HLneoh commented Apr 28, 2023

System information

Ubuntu 22.04

Virtual env #1
image

Virtual env #2
image

Description

Profiler only generates xplane.pb. Missing plugin files for Overview Page, Input Pipeline Analyzer, TensorFlow Stats, Trace Viewer, GPU Kernel Stats, Memory Profile Tool and Pod Viewer.

Results

Warning obtained as below when viewing the overview page.
No step marker observed and hence the step time is unknown. This may happen if (1) training steps are not instrumented (e.g., if you are not using Keras) or (2) the profiling duration is shorter than the step time. For (1), you need to add step instrumentation; for (2), you may try to profile longer.

image

@zzzaries
Copy link
Collaborator

zzzaries commented May 2, 2023

Hi HLneoh, just want to check that if the tools (eg. overview page, tensorflow stats..etc) still works for you? As if they still works fine for you, this should be something expected. Because our recent change removed dependency on those materialized JSON tool files, but the data will be generated from xplane.pb on the fly directly.

@martintx
Copy link

martintx commented Jun 6, 2023

Hi @HLneoh, I have an environment that is similar to your env2, but it seems that due to issues with protobuf, the profile page is not displayed at all. It would be greatly appreciated if you could share your protobuf version with me, tks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants
@martintx @zzzaries @HLneoh and others