{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Generate DISARM files\n",
"\n",
"Generate DISARM files and database objects from the DISARM master spreadsheets."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 1. Generate DISARM github pages"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"updated ../generated_pages/disarm_red_framework.md\n",
"updated ../generated_files/disarm_red_framework_clickable.html\n",
"updated ../generated_pages/disarm_blue_framework.md\n",
"updated ../generated_files/disarm_blue_framework_clickable.html\n",
"Temp: objecttype phase\n",
"updated ../generated_pages/phases_index.md\n",
"Temp: objecttype tactic\n",
"updated ../generated_pages/tactics_index.md\n",
"Updating ../generated_pages/tactics/TA10.md\n",
"Temp: objecttype technique\n",
"updated ../generated_pages/techniques_index.md\n",
"Updating ../generated_pages/techniques/T0103.001.md\n",
"Updating ../generated_pages/techniques/T0103.002.md\n",
"Updating ../generated_pages/techniques/T0128.md\n",
"Temp: objecttype task\n",
"updated ../generated_pages/tasks_index.md\n",
"Temp: objecttype incident\n",
"updated ../generated_pages/incidents_index.md\n",
"Temp: objecttype counter\n",
"updated ../generated_pages/counters_index.md\n",
"Temp: objecttype metatechnique\n",
"updated ../generated_pages/metatechniques_index.md\n",
"Temp: objecttype actortype\n",
"updated ../generated_pages/actortypes_index.md\n",
"updated ../generated_pages/responsetype_index.md\n",
"updated ../generated_pages/detections_index.md\n",
"updated ../generated_pages/tactics_by_responsetype_table.md\n",
"updated ../generated_pages/metatechniques_by_responsetype_table.md\n"
]
}
],
"source": [
"import pandas as pd\n",
"from generate_DISARM_pages import Disarm\n",
"disarm = Disarm()\n",
"disarm.generate_and_write_datafiles()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 2. Generate DISARM STIX objects"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"import sys\n",
"sys.path.insert(0, \"DISARM-STIX2\")\n",
"from main import generate_disarm_stix\n",
"\n",
"generate_disarm_stix()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 3. Generate DISARM database objects"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"from generate_disarm_sql import generate_disarm_sql\n",
"\n",
"generate_disarm_sql('sqlite')\n",
"generate_disarm_sql('postgresql_local')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 4 (optional). Look at datasets"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>disarm_id</th>\n",
" <th>name</th>\n",
" <th>summary</th>\n",
" <th>sector_ids</th>\n",
" <th>framework_ids</th>\n",
" <th>longname</th>\n",
" <th>Jon's comments/questions</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>A001</td>\n",
" <td>data scientist</td>\n",
" <td>Person who can wrangle data, implement machine...</td>\n",
" <td>S001, S002, S003, S004, S005, S006, S007, S008...</td>\n",
" <td>FW01, FW02</td>\n",
" <td>A001 - data scientist</td>\n",
" <td>What actual actions do data scientists execute...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>A002</td>\n",
" <td>target</td>\n",
" <td>Person being targeted by disinformation campaign</td>\n",
" <td>S001, S002, S003, S004, S005, S006, S007, S008...</td>\n",
" <td>FW02</td>\n",
" <td>A002 - target</td>\n",
" <td>Both red and blue framework users will refer t...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>A003</td>\n",
" <td>trusted authority</td>\n",
" <td>Influencer</td>\n",
" <td>S001, S002, S003, S004, S005, S006, S007, S008...</td>\n",
" <td>FW01, FW02</td>\n",
" <td>A003 - trusted authority</td>\n",
" <td>Is A003 best summarised by the word \"Inluencer...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>A004</td>\n",
" <td>activist</td>\n",
" <td></td>\n",
" <td>S002</td>\n",
" <td>FW02</td>\n",
" <td>A004 - activist</td>\n",
" <td>What does this actortype do?</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>A005</td>\n",
" <td>community group</td>\n",
" <td></td>\n",
" <td>S002</td>\n",
" <td>FW02</td>\n",
" <td>A005 - community group</td>\n",
" <td>What does this actortype do?</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>A006</td>\n",
" <td>educator</td>\n",
" <td></td>\n",
" <td>S002</td>\n",
" <td>FW02</td>\n",
" <td>A006 - educator</td>\n",
" <td></td>\n",
" </tr>\n",
" <tr>\n",
" <th>6</th>\n",
" <td>A007</td>\n",
" <td>factchecker</td>\n",
" <td>Someone with the skills to verify whether info...</td>\n",
" <td>S002</td>\n",
" <td>FW02</td>\n",
" <td>A007 - factchecker</td>\n",
" <td></td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>A008</td>\n",
" <td>library</td>\n",
" <td></td>\n",
" <td>S002</td>\n",
" <td>FW02</td>\n",
" <td>A008 - library</td>\n",
" <td>Is this an actortype? A sub-sector, yes. But w...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>A009</td>\n",
" <td>NGO</td>\n",
" <td></td>\n",
" <td>S002</td>\n",
" <td>FW02</td>\n",
" <td>A009 - NGO</td>\n",
" <td>This is a sector, not an actortype</td>\n",
" </tr>\n",
" <tr>\n",
" <th>9</th>\n",
" <td>A010</td>\n",
" <td>religious organisation</td>\n",
" <td></td>\n",
" <td>S002</td>\n",
" <td>FW02</td>\n",
" <td>A010 - religious organisation</td>\n",
" <td>\"Faith communities\" are a sub-sector of the Ci...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>10</th>\n",
" <td>A011</td>\n",
" <td>school</td>\n",
" <td></td>\n",
" <td>S002</td>\n",
" <td>FW02</td>\n",
" <td>A011 - school</td>\n",
" <td>This is a subsector of S004 - not an actortype...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>11</th>\n",
" <td>A012</td>\n",
" <td>account owner</td>\n",
" <td>Anyone who owns an account online</td>\n",
" <td>S006</td>\n",
" <td>FW01\\nFW02</td>\n",
" <td>A012 - account owner</td>\n",
" <td></td>\n",
" </tr>\n",
" <tr>\n",
" <th>12</th>\n",
" <td>A013</td>\n",
" <td>content creator</td>\n",
" <td></td>\n",
" <td>S006</td>\n",
" <td>FW01\\nFW02</td>\n",
" <td>A013 - content creator</td>\n",
" <td></td>\n",
" </tr>\n",
" <tr>\n",
" <th>13</th>\n",
" <td>A014</td>\n",
" <td>elves</td>\n",
" <td></td>\n",
" <td>S006</td>\n",
" <td>FW02</td>\n",
" <td>A014 - elves</td>\n",
" <td>??? SJ, you have told me, but I've forgotten</td>\n",
" </tr>\n",
" <tr>\n",
" <th>14</th>\n",
" <td>A015</td>\n",
" <td>general public</td>\n",
" <td></td>\n",
" <td>S006</td>\n",
" <td>FW02</td>\n",
" <td>A015 - general public</td>\n",
" <td>Duplication of a field in the sector object. B...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>A016</td>\n",
" <td>influencer</td>\n",
" <td></td>\n",
" <td>S006</td>\n",
" <td>FW01\\nFW02</td>\n",
" <td>A016 - influencer</td>\n",
" <td></td>\n",
" </tr>\n",
" <tr>\n",
" <th>16</th>\n",
" <td>A017</td>\n",
" <td>coordinating body</td>\n",
" <td>For example the DHS</td>\n",
" <td>S003</td>\n",
" <td>FW02</td>\n",
" <td>A017 - coordinating body</td>\n",
" <td>\"Response coordinator\"? Or just 'Coordinator\"</td>\n",
" </tr>\n",
" <tr>\n",
" <th>17</th>\n",
" <td>A018</td>\n",
" <td>government</td>\n",
" <td>Government agencies</td>\n",
" <td>S003</td>\n",
" <td>FW01\\nFW02</td>\n",
" <td>A018 - government</td>\n",
" <td>DHS (A017) is \"government\". Again, this is a f...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>18</th>\n",
" <td>A019</td>\n",
" <td>military</td>\n",
" <td></td>\n",
" <td>S003</td>\n",
" <td>FW02</td>\n",
" <td>A019 - military</td>\n",
" <td>A019 is a sub-sector of \"government\". Again, t...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>19</th>\n",
" <td>A020</td>\n",
" <td>policy maker</td>\n",
" <td></td>\n",
" <td>S003</td>\n",
" <td>FW02</td>\n",
" <td>A020 - policy maker</td>\n",
" <td></td>\n",
" </tr>\n",
" <tr>\n",
" <th>20</th>\n",
" <td>A021</td>\n",
" <td>media organisation</td>\n",
" <td></td>\n",
" <td>S010</td>\n",
" <td>FW01\\nFW02</td>\n",
" <td>A021 - media organisation</td>\n",
" <td>Not an actortype. The actor is perhaps as the ...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>A022</td>\n",
" <td>company</td>\n",
" <td></td>\n",
" <td>S009</td>\n",
" <td>FW02</td>\n",
" <td>A022 - company</td>\n",
" <td></td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>A023</td>\n",
" <td>adtech provider</td>\n",
" <td></td>\n",
" <td>S008</td>\n",
" <td>FW02</td>\n",
" <td>A023 - adtech provider</td>\n",
" <td>Should Adtech be a called-out subsector of S00...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>23</th>\n",
" <td>A024</td>\n",
" <td>developer</td>\n",
" <td></td>\n",
" <td>S008</td>\n",
" <td>FW02</td>\n",
" <td>A024 - developer</td>\n",
" <td></td>\n",
" </tr>\n",
" <tr>\n",
" <th>24</th>\n",
" <td>A025</td>\n",
" <td>funding_site_admin</td>\n",
" <td>Funding site admin</td>\n",
" <td>S008</td>\n",
" <td>FW02</td>\n",
" <td>A025 - funding_site_admin</td>\n",
" <td>What is this?</td>\n",
" </tr>\n",
" <tr>\n",
" <th>25</th>\n",
" <td>A026</td>\n",
" <td>games designer</td>\n",
" <td></td>\n",
" <td>S008</td>\n",
" <td>FW01, FW02</td>\n",
" <td>A026 - games designer</td>\n",
" <td>Just \"Designer\"? Would the TTP context not mak...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>A027</td>\n",
" <td>information security</td>\n",
" <td></td>\n",
" <td>S008</td>\n",
" <td>FW02</td>\n",
" <td>A027 - information security</td>\n",
" <td></td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>A028</td>\n",
" <td>platform administrator</td>\n",
" <td></td>\n",
" <td>S008</td>\n",
" <td>FW02</td>\n",
" <td>A028 - platform administrator</td>\n",
" <td>Just \"Administrator\"?</td>\n",
" </tr>\n",
" <tr>\n",
" <th>28</th>\n",
" <td>A029</td>\n",
" <td>server admininistrator</td>\n",
" <td></td>\n",
" <td>S008</td>\n",
" <td>FW02</td>\n",
" <td>A029 - server admininistrator</td>\n",
" <td>Just \"Administrator\"? (de-duplicating A028)</td>\n",
" </tr>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>A030</td>\n",
" <td>platforms</td>\n",
" <td></td>\n",
" <td>S007</td>\n",
" <td>FW02</td>\n",
" <td>A030 - platforms</td>\n",
" <td>The same as S007? If there is an action here, ...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>30</th>\n",
" <td>A031</td>\n",
" <td>social media platform adminstrator</td>\n",
" <td>Person with the authority to make changes to a...</td>\n",
" <td>S007</td>\n",
" <td>FW02</td>\n",
" <td>A031 - social media platform adminstrator</td>\n",
" <td>Not needed, given A028. The sector selection d...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>A032</td>\n",
" <td>social media platform outreach</td>\n",
" <td></td>\n",
" <td>S007</td>\n",
" <td>FW02</td>\n",
" <td>A032 - social media platform outreach</td>\n",
" <td>What does this do?</td>\n",
" </tr>\n",
" <tr>\n",
" <th>32</th>\n",
" <td>A033</td>\n",
" <td>social media platform owner</td>\n",
" <td>Person with authority to make changes to a soc...</td>\n",
" <td>S007</td>\n",
" <td>FW02</td>\n",
" <td>A033 - social media platform owner</td>\n",
" <td></td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" disarm_id name \\\n",
"0 A001 data scientist \n",
"1 A002 target \n",
"2 A003 trusted authority \n",
"3 A004 activist \n",
"4 A005 community group \n",
"5 A006 educator \n",
"6 A007 factchecker \n",
"7 A008 library \n",
"8 A009 NGO \n",
"9 A010 religious organisation \n",
"10 A011 school \n",
"11 A012 account owner \n",
"12 A013 content creator \n",
"13 A014 elves \n",
"14 A015 general public \n",
"15 A016 influencer \n",
"16 A017 coordinating body \n",
"17 A018 government \n",
"18 A019 military \n",
"19 A020 policy maker \n",
"20 A021 media organisation \n",
"21 A022 company \n",
"22 A023 adtech provider \n",
"23 A024 developer \n",
"24 A025 funding_site_admin \n",
"25 A026 games designer \n",
"26 A027 information security \n",
"27 A028 platform administrator \n",
"28 A029 server admininistrator \n",
"29 A030 platforms \n",
"30 A031 social media platform adminstrator \n",
"31 A032 social media platform outreach \n",
"32 A033 social media platform owner \n",
"\n",
" summary \\\n",
"0 Person who can wrangle data, implement machine... \n",
"1 Person being targeted by disinformation campaign \n",
"2 Influencer \n",
"3 \n",
"4 \n",
"5 \n",
"6 Someone with the skills to verify whether info... \n",
"7 \n",
"8 \n",
"9 \n",
"10 \n",
"11 Anyone who owns an account online \n",
"12 \n",
"13 \n",
"14 \n",
"15 \n",
"16 For example the DHS \n",
"17 Government agencies \n",
"18 \n",
"19 \n",
"20 \n",
"21 \n",
"22 \n",
"23 \n",
"24 Funding site admin \n",
"25 \n",
"26 \n",
"27 \n",
"28 \n",
"29 \n",
"30 Person with the authority to make changes to a... \n",
"31 \n",
"32 Person with authority to make changes to a soc... \n",
"\n",
" sector_ids framework_ids \\\n",
"0 S001, S002, S003, S004, S005, S006, S007, S008... FW01, FW02 \n",
"1 S001, S002, S003, S004, S005, S006, S007, S008... FW02 \n",
"2 S001, S002, S003, S004, S005, S006, S007, S008... FW01, FW02 \n",
"3 S002 FW02 \n",
"4 S002 FW02 \n",
"5 S002 FW02 \n",
"6 S002 FW02 \n",
"7 S002 FW02 \n",
"8 S002 FW02 \n",
"9 S002 FW02 \n",
"10 S002 FW02 \n",
"11 S006 FW01\\nFW02 \n",
"12 S006 FW01\\nFW02 \n",
"13 S006 FW02 \n",
"14 S006 FW02 \n",
"15 S006 FW01\\nFW02 \n",
"16 S003 FW02 \n",
"17 S003 FW01\\nFW02 \n",
"18 S003 FW02 \n",
"19 S003 FW02 \n",
"20 S010 FW01\\nFW02 \n",
"21 S009 FW02 \n",
"22 S008 FW02 \n",
"23 S008 FW02 \n",
"24 S008 FW02 \n",
"25 S008 FW01, FW02 \n",
"26 S008 FW02 \n",
"27 S008 FW02 \n",
"28 S008 FW02 \n",
"29 S007 FW02 \n",
"30 S007 FW02 \n",
"31 S007 FW02 \n",
"32 S007 FW02 \n",
"\n",
" longname \\\n",
"0 A001 - data scientist \n",
"1 A002 - target \n",
"2 A003 - trusted authority \n",
"3 A004 - activist \n",
"4 A005 - community group \n",
"5 A006 - educator \n",
"6 A007 - factchecker \n",
"7 A008 - library \n",
"8 A009 - NGO \n",
"9 A010 - religious organisation \n",
"10 A011 - school \n",
"11 A012 - account owner \n",
"12 A013 - content creator \n",
"13 A014 - elves \n",
"14 A015 - general public \n",
"15 A016 - influencer \n",
"16 A017 - coordinating body \n",
"17 A018 - government \n",
"18 A019 - military \n",
"19 A020 - policy maker \n",
"20 A021 - media organisation \n",
"21 A022 - company \n",
"22 A023 - adtech provider \n",
"23 A024 - developer \n",
"24 A025 - funding_site_admin \n",
"25 A026 - games designer \n",
"26 A027 - information security \n",
"27 A028 - platform administrator \n",
"28 A029 - server admininistrator \n",
"29 A030 - platforms \n",
"30 A031 - social media platform adminstrator \n",
"31 A032 - social media platform outreach \n",
"32 A033 - social media platform owner \n",
"\n",
" Jon's comments/questions \n",
"0 What actual actions do data scientists execute... \n",
"1 Both red and blue framework users will refer t... \n",
"2 Is A003 best summarised by the word \"Inluencer... \n",
"3 What does this actortype do? \n",
"4 What does this actortype do? \n",
"5 \n",
"6 \n",
"7 Is this an actortype? A sub-sector, yes. But w... \n",
"8 This is a sector, not an actortype \n",
"9 \"Faith communities\" are a sub-sector of the Ci... \n",
"10 This is a subsector of S004 - not an actortype... \n",
"11 \n",
"12 \n",
"13 ??? SJ, you have told me, but I've forgotten \n",
"14 Duplication of a field in the sector object. B... \n",
"15 \n",
"16 \"Response coordinator\"? Or just 'Coordinator\" \n",
"17 DHS (A017) is \"government\". Again, this is a f... \n",
"18 A019 is a sub-sector of \"government\". Again, t... \n",
"19 \n",
"20 Not an actortype. The actor is perhaps as the ... \n",
"21 \n",
"22 Should Adtech be a called-out subsector of S00... \n",
"23 \n",
"24 What is this? \n",
"25 Just \"Designer\"? Would the TTP context not mak... \n",
"26 \n",
"27 Just \"Administrator\"? \n",
"28 Just \"Administrator\"? (de-duplicating A028) \n",
"29 The same as S007? If there is an action here, ... \n",
"30 Not needed, given A028. The sector selection d... \n",
"31 What does this do? \n",
"32 "
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"pd.set_option('display.max_rows', 1000)\n",
"disarm.df_actortypes"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.3"
}
},
"nbformat": 4,
"nbformat_minor": 4
}