group_references missing

[flowtrader2016]

Hello,

Thanks for creating the ATTACK-Python-Client as this is very useful. I have observed the following:

If I create a group variable

groups = all_attack['groups']

and then a Pandas Dataframe from the groups data

df = json_normalize(groups)

I can then see the group_references associated with the "Lazarus Group"

df[df.group == "Lazarus Group"].group_references.values[0]

['https://attack.mitre.org/wiki/Group/G0032',
 'https://www.us-cert.gov/ncas/alerts/TA17-164A',
 'https://www.operationblockbuster.com/wp-content/uploads/2016/02/Operation-Blockbuster-Report.pdf']

However, when I go to the page https://attack.mitre.org/wiki/Group/G0032 I see there are 11 references and not the 3 returned by ATTACK-Python-Client.

Is this a bug?

Thanks

[Cyb3rWard0g]

Hey @flowtrader2016 ! Thank you for opening this issue. I know what the problem is. I parse the main reference values from the main "intrusion-set" object ONLY. Intrusion-set translates to Group. so for example Lazarus has the following:

{
    "objects": [
        {
            "name": "Lazarus Group",
            "created": "2017-05-31T21:32:03.807Z",
            "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
            "description": "Lazarus Group is a threat group that has been attributed to the North Korean government. (Citation: US-CERT HIDDEN COBRA June 2017) The group has been active since at least 2009 and was reportedly responsible for the November 2014 destructive wiper attack against Sony Pictures Entertainment as part of a campaign named Operation Blockbuster by Novetta. Malware used by Lazarus Group correlates to other reported campaigns, including Operation Flame, Operation 1Mission, Operation Troy, DarkSeoul, and Ten Days of Rain. (Citation: Novetta Blockbuster)",
            "external_references": [
                {
                    "url": "https://attack.mitre.org/wiki/Group/G0032",
                    "source_name": "mitre-attack",
                    "external_id": "G0032"
                },
                {
                    "description": "US-CERT. (2017, June 13). Alert (TA17-164A) HIDDEN COBRA \u2013 North Korea\u2019s DDoS Botnet Infrastructure. Retrieved July 13, 2017.",
                    "source_name": "US-CERT HIDDEN COBRA June 2017",
                    "url": "https://www.us-cert.gov/ncas/alerts/TA17-164A"
                },
                {
                    "description": "Novetta Threat Research Group. (2016, February 24). Operation Blockbuster: Unraveling the Long Thread of the Sony Attack. Retrieved February 25, 2016.",
                    "source_name": "Novetta Blockbuster",
                    "url": "https://www.operationblockbuster.com/wp-content/uploads/2016/02/Operation-Blockbuster-Report.pdf"
                }
            ],
            "object_marking_refs": [
                "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
            ],
            "id": "intrusion-set--c93fccb1-e8e8-42cf-ae33-2ad1d183913a",
            "modified": "2018-04-18T17:59:24.739Z",
            "aliases": [
                "Lazarus Group",
                "HIDDEN COBRA",
                "Guardians of Peace",
                "ZINC",
                "NICKEL ACADEMY"
            ],
            "type": "intrusion-set"
        }
    ],
    "type": "bundle",
    "id": "bundle--dee7ed09-5c64-441d-83ef-84c7595f9597",
    "spec_version": "2.0"
}

However, when I add relationships objects to Groups, then other extra references appear. For example, these are the a few of the relationships objects mapped to Lazarus Intrusion-set:

{
    "objects": [
        {
            "created": "2018-04-18T17:59:24.739Z",
            "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
            "object_marking_refs": [
                "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
            ],
            "description": "A Lazarus Group malware sample performs reflective DLL injection.",
            "external_references": [
                {
                    "description": "Sherstobitoff, R. (2018, February 12). Lazarus Resurfaces, Targets Global Banks and Bitcoin Users. Retrieved February 19, 2018.",
                    "source_name": "McAfee Lazarus Resurfaces Feb 2018",
                    "url": "https://securingtomorrow.mcafee.com/mcafee-labs/lazarus-resurfaces-targets-global-banks-bitcoin-users/"
                }
            ],
            "modified": "2018-04-18T17:59:24.739Z",
            "source_ref": "intrusion-set--c93fccb1-e8e8-42cf-ae33-2ad1d183913a",
            "relationship_type": "uses",
            "target_ref": "attack-pattern--43e7dc91-05b2-474c-b9ac-2ed4fe101f4d",
            "type": "relationship",
            "id": "relationship--c055d228-a0e6-4b76-961a-1eebe99679f5"
        }
    ],
    "type": "bundle",
    "id": "bundle--055d1be8-9dba-49e6-961d-3eb71a0b941a",
    "spec_version": "2.0"
}

{
    "objects": [
        {
            "created": "2017-05-31T21:33:27.067Z",
            "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
            "object_marking_refs": [
                "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
            ],
            "description": "Lazarus Group malware IndiaIndia checks Registry keys within HKCU and HKLM to determine if certain applications are present, including SecureCRT, Terminal Services, RealVNC, TightVNC, UltraVNC, Radmin, mRemote, TeamViewer, FileZilla, pcAnyware, and Remote Desktop.Lazarus Group malware sample checks for the presence of the following Registry key:<code>HKEY_CURRENT_USER\\Software\\Bitcoin\\Bitcoin-Qt</code>.",
            "external_references": [
                {
                    "description": "Novetta Threat Research Group. (2016, February 24). Operation Blockbuster: Unraveling the Long Thread of the Sony Attack. Retrieved February 25, 2016.",
                    "source_name": "Novetta Blockbuster",
                    "url": "https://www.operationblockbuster.com/wp-content/uploads/2016/02/Operation-Blockbuster-Report.pdf"
                },
                {
                    "description": "Novetta Threat Research Group. (2016, February 24). Operation Blockbuster: Loaders, Installers and Uninstallers Report. Retrieved March 2, 2016.",
                    "source_name": "Novetta Blockbuster Loaders",
                    "url": "https://operationblockbuster.com/wp-content/uploads/2016/02/Operation-Blockbuster-Loaders-Installers-and-Uninstallers-Report.pdf"
                },
                {
                    "description": "Sherstobitoff, R. (2018, February 12). Lazarus Resurfaces, Targets Global Banks and Bitcoin Users. Retrieved February 19, 2018.",
                    "source_name": "McAfee Lazarus Resurfaces Feb 2018",
                    "url": "https://securingtomorrow.mcafee.com/mcafee-labs/lazarus-resurfaces-targets-global-banks-bitcoin-users/"
                }
            ],
            "id": "relationship--39b735d3-c659-4d1a-8e7e-082c0f049c2d",
            "modified": "2018-04-18T17:59:24.739Z",
            "source_ref": "intrusion-set--c93fccb1-e8e8-42cf-ae33-2ad1d183913a",
            "relationship_type": "uses",
            "target_ref": "attack-pattern--c32f7008-9fea-41f7-8366-5eb9b74bd896",
            "type": "relationship"
        }
    ],
    "type": "bundle",
    "id": "bundle--4156e90f-a36e-48d5-ab9f-97cf59aba066",
    "spec_version": "2.0"
}

Currently, I do not pull references from relationships:

https://github.com/Cyb3rWard0g/ATTACK-Python-Client/blob/master/attackcti/attack_api.py#L112

I will add that in the next update. Thank you 👍. Other than that, let me know if there is anything else I might be missing there 👍

[flowtrader2016]

Thank you for the explanation and quick response 👍