Unable to create local link (XCP-NG + CS 4.15 + PreSetup SR)

[edikevich]

ISSUE TYPE

• Bug Report

COMPONENT NAME

Hypervisor

CLOUDSTACK VERSION

4.15 (fresh install)

CONFIGURATION

Management Server : CentOS 7.9 (also tested on CentOS 8.2, Oracle Linux 8.3)
Hypervisor : 2 x XCP-ng 7.6 (also tested on 8.1, 8.2, in the same pool)
Zone : Advanced without SG PreSetup SR

SUMMARY

On a freshly installed CS 4.15 I can't deploy zone with presetup iscsi storage. Unable to create the local link network. With NFS deploy was complited. On CS 4.14 my configuration works well.

STEPS TO REPRODUCE

Add an XCP-ng 7.6 (8.1, 8.2) host under an Advanced Zone with PreSetup iSCSI storage.

EXPECTED RESULTS

Local link creation should be successful.

ACTUAL RESULTS

2021-02-02 19:56:04,663 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-7:ctx-090ea653) (logid:e138cefc) Seq 2-9037880027202453511: Executing request
2021-02-02 19:56:04,671 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-7:ctx-090ea653) (logid:8543a8cb) Can't find a vif on dom0 for link local, creating a new one
2021-02-02 19:56:04,673 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-7:ctx-090ea653) (logid:8543a8cb) Lowest available Vif device number: 0 for VM: Control domain on host: xcp-ng76-u4037-02
2021-02-02 19:56:04,677 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-7:ctx-090ea653) (logid:8543a8cb) Unable to create local link network
The server failed to handle your request, due to an internal error.  The given message may give details useful for debugging the problem.
        at com.xensource.xenapi.Types.checkResponse(Types.java:1609)
        at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
        at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:457)
        at com.xensource.xenapi.VIF.create(VIF.java:319)
        at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.setupLinkLocalNetwork(CitrixResourceBase.java:4822)
        at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixReadyCommandWrapper.execute(CitrixReadyCommandWrapper.java:52)
        at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixReadyCommandWrapper.execute(CitrixReadyCommandWrapper.java:38)
        at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
        at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1759)
        at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
2021-02-02 19:56:04,678 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-7:ctx-090ea653) (logid:8543a8cb) Seq 2-9037880027202453511: Response Received:
2021-02-02 19:56:04,678 DEBUG [c.c.a.t.Request] (DirectAgent-7:ctx-090ea653) (logid:8543a8cb) Seq 2-9037880027202453511: Processing:  { Ans: , MgmtId: 236109816422411, via: 2(xcp-ng76-u4037-02), Ver: v1, Flags: 110, [{"com.cloud.agent.api.UnsupportedAnswer":{"result":"false","details":"Unsupported command issued: com.cloud.agent.api.ReadyCommand.  Are you sure you got the right type of server?","wait":"0"}}] }
2021-02-02 19:56:04,678 DEBUG [c.c.a.m.AgentAttache] (DirectAgent-7:ctx-090ea653) (logid:8543a8cb) Seq 2-9037880027202453511: No more commands found
2021-02-02 19:56:04,678 DEBUG [c.c.a.t.Request] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Seq 2-9037880027202453511: Received:  { Ans: , MgmtId: 236109816422411, via: 2(xcp-ng76-u4037-02), Ver: v1, Flags: 110, { UnsupportedAnswer } }
2021-02-02 19:56:04,678 WARN  [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Unsupported Command: Unsupported command issued: com.cloud.agent.api.ReadyCommand.  Are you sure you got the right type of server?
2021-02-02 19:56:04,678 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Details from executing class com.cloud.agent.api.ReadyCommand: Unsupported command issued: com.cloud.agent.api.ReadyCommand.  Are you sure you got the right type of server?
2021-02-02 19:56:04,679 INFO  [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Host 2 is disconnecting with event AgentDisconnected
2021-02-02 19:56:04,680 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) The next status of agent 2is Alert, current status is Connecting
2021-02-02 19:56:04,680 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Deregistering link for 2 with state Alert
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Remove Agent : 2
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.DirectAgentAttache] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Processing disconnect 2(xcp-ng76-u4037-02)
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Sending Disconnect to listener: com.cloud.hypervisor.xenserver.discoverer.XcpServerDiscoverer
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Sending Disconnect to listener: com.cloud.hypervisor.hyperv.discoverer.HypervServerDiscoverer
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Sending Disconnect to listener: com.cloud.vm.ClusteredVirtualMachineManagerImpl
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Sending Disconnect to listener: com.cloud.hypervisor.vmware.manager.VmwareManagerImpl
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Sending Disconnect to listener: com.cloud.deploy.DeploymentPlanningManagerImpl
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Sending Disconnect to listener: com.cloud.storage.secondary.SecondaryStorageListener
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Sending Disconnect to listener: com.cloud.network.security.SecurityGroupListener
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Sending Disconnect to listener: com.cloud.storage.listener.StoragePoolMonitor

I am able to experiment in my test environment as needed (including CS builds/reinstall or XCP reinstalls).

[shwstppr]

@edikevich did you face this issue only with an iscsi presetup SR and not with a NFS presetup SR? I tried with ACS 4.15, xcp-ng8.1 and a NFS presetup SR as primary store and zone worked fine.
@rhtyd @andrijapanicsb anything specific you guys can think of which can cause this?

[edikevich]

Hi. Yes, I have issue only with iSCSI presetup SR

[Pearl1594]

@edikevich I wasn't able to reproduce the issue. I have the following SR on the host

Following were the details added for primary store during zone deployment

And I was able to successfully deploy the zone:

However, I did notice that the zone deployment view in the new UI has an issue wrt adding the Primary store with presetup protocol which has been addressed with #4845

If you still notice an issue, can you please let us know at which phase of zone deployment do you see a failure. Thanks!

[edikevich]

Hi. I will try again on next week. I will let you know If I have a problem.