horizon, max_episode_length and CBF of locomotion

[lch806376525-sketch]

Thank you for your reply. I checked the locomotion branch of the diffuser project, and the horizon setting is indeed 32. I suspect you set the horizon to 600 to obtain the visualization image of the longer-horizon trajectory (Figure 3), is that correct?

Furthermore, I also noticed that you modified the max_episode_length to 500 (it is 1000 in the original diffuser project). So, which max_episode_length were used to calculate the SCORE and other data in Table 3: Robot safe planning comparisons with benchmarks? Thank you for the clarification. Meanwhile, is the diffusion model used to obtain these data using the horizon=32 diffusion model or the modified 600 diffusion model.

Besides, in the locomotion branch's ROS diffuser, why is the robustness term of 0.01 not included in the CBF (i.e., b = height - x[:,6:7] # - 0.1*x[:,15:16] # - 0.01 # for robustness)? Shouldn't this robustness term be necessary to ensure the effectiveness of the ROS diffuser? Your ROS related codes are as follows:
`@torch.no_grad() #only for sampling
def invariance(self, x, xp1): # RoS diffuser

    x = x.squeeze(0)
    xp1 = xp1.squeeze(0)

    nBatch = x.shape[0]
    ref = xp1 - x

    #normalize obstacle: Gaussian, x:0-6 control, 6-23 state
    height = 1.3
    height = (height - self.mean[0]) / self.std[0]

    #CBF
    ############################################ceiling
    b = height - x[:,6:7] # - 0.1*x[:,15:16]   # - 0.01  # for robustness
    Lfb = 0 
    Lgbu1 = -1*torch.ones_like(x[:,6:7])
    #Lgbu2 = -0.1*torch.ones_like(x[:,6:7])

    G = torch.cat([-Lgbu1], dim = 1)
    G = G.unsqueeze(1)
    k = 1
    h = Lfb + k*b
    

    q = -torch.cat([ref[:,6:7]], dim = 1).to(G.device)  #, ref[:,15:16]
    Q = Variable(torch.eye(1))
    Q = Q.unsqueeze(0).expand(nBatch, 1, 1).to(G.device)
    
    e = Variable(torch.Tensor())
    out = QPFunction(verbose=-1, solver = QPSolvers.PDIPM_BATCHED)(Q, q, G, h, e, e)

    rt = xp1.clone()      
    rt[:,6:7] = x[:,6:7] + out[:,0:1]
    # rt[:,15:16] = x[:,15:16] + out[:,1:2]
    # print(out[0:4,0:1])
    rt = rt.unsqueeze(0)
    return rt, torch.min(b)  # + 0.01  # for robustness`

Thank you once again for your guidance.

Originally posted by @lch806376525-sketch in #17

[Weixy21]

Yes, I may have changed the horizon to 600. You can also do similar things by either training a new model or using existing pre-trained models (if the model cannot load, you may debug by adjusting the U-net).

max_episode_length 500 is used to get the score, the horizon is all 600 in the experiments.

In locomotion, the safety (due to inter-sampling) is not sensitive to the boundaries (lines instead of super-ellipses in maze), so I did not add robustness.

[lch806376525-sketch]

Thank you for your clarification.
I still have some doubts about the issue of CBF. I believe that if ROS removes this robust term, the theory of CBF cannot guarantee that the trajectories that are initially not in the safe set will eventually be in the safe set. (Because the guidance for this robust term is obtained through Lyapunov analysis).
Since you mentioned that this issue may be related to the type of boundary, could you please further explain the reasons for this, or in fact, this small robust term does not actually reflect its function under simple linear boundaries?
Anyway, should I just add the robust term 0.01 to make it theoretically more reliable?

[Weixy21]

Yes, theoretically, the CBF cannot guarantee safety if the safety constraint is initially violated (due to the Lyapunov asymptotic property). However, since there are uncertainties in diffusion, as well as due to inter-sampling effect (different boundaries may have different sensitivities), the state may actually enter the safe set within finite time.

You can add a robust term to make it theoretically more reliable (and the robust value can be theoretically got from the theorem in the paper).

[lch806376525-sketch]

Yes, I may have changed the horizon to 600. You can also do similar things by either training a new model or using existing pre-trained models (if the model cannot load, you may debug by adjusting the U-net).

May I ask if you have reserved the pretrained model with horizon=600 for walker2d and hopper？ And can you share them.

[Weixy21]

I have limited access to my workstation as I am travelling. I will check when I am back. In the meantime, I would encourage you to train it by yourself. It won't take much time

[lch806376525-sketch]

I have limited access to my workstation as I am travelling. I will check when I am back. In the meantime, I would encourage you to train it by yourself. It won't take much time

Thank you for your help and guidance. I am currently trying to train, but due to my limited computing power, I expect the training time to be much longer than indicated in the paper. However, I will do my best to try.
Have a pleasant journey！