About Behavior 1k: the task can not be displayed completely

[ClorisUcas]

Describe the bug

Thanks for your great work !
I am trying to tun the task "putting_meal_on_plate" in Behavior 1k, but the simulator cannot show the task completely.
It seems like something wring in my configuration. Hope to get your reply and answer, thanks again:)

**My running file"
test.txt

"Log"
log.txt

Desktop (please complete the following information):

• OS: [e.g. Ubuntu 22.04]
• Isaac Sim Version [e.g. 4.1.0]
• OmniGibson Version [e.g. 1.1.1]

[hang-yin]

Hi @ClorisUcas , I just tried running your script, but everything looks fine on my end. What exactly do you mean by "cannot show the task completely"? Do you mean that you can not see the scene?

There are two viewports, one as the main world camera, and the other on the robot. It seems like the robot camera is blocking the world camera a little on my end. A simple fix for this is you can set a breakpoint right after you get the robot observation, and then run for _ in range(1000): og.sim.render(). Once that line is running, you can use the isaac sim ui and drag the robot viewport to the side so that you can see the main viewport.

As a sanity check, you can also take a look at the robot observation at the breakpoint.

Please let me know how it goes!

[hang-yin]

Hi @ClorisUcas , I see your point!

We are not yet at a point where we have robot trajectories to complete every task. However, we are actively working on curating this dataset. Loading the task into omnigibson only instantiates it, but not actually execute it. If you want to try, we do have tools such as the physical action primitives and symbolic primitives to execture segments of these tasks. This is somewhat an experimental feature though, we don't expect it to work 100% of the time.

[ClorisUcas]

Thank you for your helpful response. So I think some tasks like "make a salad" also need the user to instantiate a "salad" in the scene?

[hang-yin]

@ClorisUcas here's the task definition in our knowledgebase: https://behavior.stanford.edu/knowledgebase/tasks/make_a_salad-0/index.html

Instantiating the task in a scene only gives you the initial condition necessary to complete this task. For example, in this task, you would get all the ingredients and furnitures needed to make a salad. The salad itself, however, is the combination of everything as specified in the BDDL goal condition.

[ClorisUcas]

Thank you for your detailed explanation that gives me a deeper undersranding of the behavior1k project.
However, I encounter a new issue of "ValueError: invalid literal for int() with base 10: '0.0'" when trying to render the "make a salad" scene. The log.txt gives detailed information.
Could you help me look into the cause? ღ( ´･ᴗ･` )

[hang-yin]

@ClorisUcas this is interesting. Do you mind sending a repro script for this? I can run it on my end to see what is going wrong.