ydkhatri / mac_apt

macOS (& ios) Artifact Parsing Tool

mac_apt - macOS (and iOS) Artifact Parsing Tool

mac_apt is a DFIR (Digital Forensics and Incident Response) tool to process Mac computer full disk images (or live machines) and extract data/metadata useful for forensic investigation. It is a python based framework, which has plugins to process individual artifacts (such as Safari internet history, Network interfaces, Recently accessed files & volumes, ..)

mac_apt now also includes ios_apt, for processing ios images.

Requirements: Python 3.9 or above (64 bit)

Note: Tested upto Python 3.12 on Windows and macOS.

Features

• Cross platform (no dependency on pyobjc)
• Works on E01, VMDK, AFF4, DD, split-DD, DMG (no compression), SPARSEIMAGE & mounted images
• XLSX, CSV, TSV, Sqlite outputs
• Analyzed files/artifacts are exported for later review
• zlib, lzvn, lzfse compressed files are supported!
• Native HFS & APFS parser
• Reads the Spotlight database and Unified Logging (tracev3) files

Latest

✔️ Can read Axiom created targeted collection zip files
✔️ ios_apt can read GrayKey extracted file system
✔️ Can read RECON and ASLA created .sparseimage files
✔️ Support for macOS Big Sur Sealed volumes (11.0)
✔️ Introducing ios_apt for processing iOS/ipadOS images
✔️ FAST mode ⏳
✔️ Encrypted 🔒 APFS images can now be processed using password/recovery-key 🔑
✔️ macOS Catalina (10.15+) separately mounted SYSTEM & DATA volumes now supported
✔️ AFF4 images (including macquisition created) are supported

Available Plugins (artifacts parsed)	Description
APPLIST	Reads apps & printers installed and/or available for each user from appList.dat
ARD	Reads ARD (Apple Remote Desktop) cached databases about app usage
ASL	Reads ASL (Apple System Log) from asl.log, asl.db and ".asl" files
AUTOSTART	Retrieves programs, daemons, services set to start at boot/login
BASICINFO	Basic machine & OS configuration like SN, timezone, computer name, last logged in user, HFS info
BLUETOOTH	Gets Bluetooth Artifacts
CHROMIUM	Read Chromium Browsers (Edge, Chrome, Opera,..) History, Top Sites, Downloads and Extension info
CFURLCACHE	Reads CFURL cache to URLs, requests and responses
COOKIES	Reads .binarycookies, .cookies files and HSTS.plist for each user
DOCKITEMS	Reads the Dock plist for every user
DOCUMENTREVISIONS	Reads DocumentRevisions database
DOMAINS	Active Directory Domain(s) that the mac is connected to
FILESHARING	Read shared folder info
FIREFOX	Read internet history from Mozilla Firefox browser
FSEVENTS	Reads file system event logs (from .fseventsd)
IDEVICEBACKUPS	Reads and exports iPhone/iPad backup databases
IDEVICEINFO	Reads and exports connected iDevice details
IMESSAGE	Read iMessage chats
INETACCOUNTS	Retrieve configured internet accounts (iCloud, Google, Linkedin, facebook..)
INSTALLHISTORY	Software Installation History
MSOFFICE	Reads Word, Excel, Powerpoint and other office MRU/accessed file paths
MSRDC	Reads connection history from Microsoft Remote Desktop database and extracts thumbnails
NETUSAGE	Read network usage data statistics per application
NETWORKING	Interfaces, last IP address, MAC address, DHCP ..
NOTES	Reads notes databases
NOTIFICATIONS	Reads mac notification data for each user
PRINTJOBS	Parses CUPS spooled print jobs to get information about files/commands sent to a printer
QUARANTINE	Reads the quarantine database and .LastGKReject file
QUICKLOOK	Reads the QuickLook index.sqlite and carves thumbnails from thumbnails.data
RECENTITEMS	Recently accessed Servers, Documents, Hosts, Volumes & Applications from .plist and .sfl files. Also gets recent searches and places for each user
SAFARI	Internet history, downloaded file information, cookies and more from Safari caches
SCREENSHARING	Reads the list of connected hosts with Screen Sharing
SAVEDSTATE	Gets window titles from Saved Application State info
SCREENTIME	Reads ScreenTime database for program and app usage
SPOTLIGHT	Reads the spotlight index databases
SPOTLIGHTSHORTCUTS	User typed data in the spotlight bar & targeted document/app
SUDOLASTRUN	Gets last time sudo was used and a few other times earlier (if available)
TCC	Reads Transparency, Consent and Control (TCC) database
TERMINALSTATE	Reads Terminal saved state files which includes full text content of terminal windows
TERMSESSIONS	Reads Terminal (bash & zsh) history & sesions for every user
UNIFIEDLOGS	Reads macOS unified logging logs from .tracev3 files
USERS	Local & Domain user information - name, UID, UUID, GID, account creation & password set dates, pass hints, homedir & Darwin paths
UTMPX	Reads utmpx file
WIFI	Gets wifi network information
XPROTECT	Reads XProtect diagnostic files and XProtect Behavior Service database

Coming soon..

• More plugins
• More documentation

For installation (to run from code) see https://github.com/ydkhatri/mac_apt/wiki/Installation-for-Python3
Please read the documentation here: https://github.com/ydkhatri/mac_apt/wiki

To download windows binaries, proceed here - https://github.com/ydkhatri/mac_apt/releases

Bugs

Feel free to send comments and feedback to [email protected], or open an issue.

mac_apt's Issues

Could not find a partition having a macOS installation on it

I succesfully took an image with MacQuisition of a hdd 1TB (disk0)
APFS is configured but not locked/encrypted.

Importing the image in X-Ways goes as expected.

However, when executing mac_apt it says that the APFS volume could not be found.
Here's the output with log on DEBUG.

2020-06-02 10:16:57|MAIN|INFO|Started macOS Artifact Parsing Tool, version 0.6
2020-06-02 10:16:57|MAIN|INFO|Dates and times are in UTC unless the specific artifact being parsed saves it as local time!
2020-06-02 10:16:57|MAIN|DEBUG|mac_apt.x64.exe -o OUT -x -l DEBUG E01 IMAGE.E01 ALL
2020-06-02 10:16:57|MAIN|INFO|Pytsk version  = 20170801
2020-06-02 10:16:57|MAIN|INFO|Pyewf version  = 20190317
2020-06-02 10:16:57|MAIN|INFO|Pyvmdk version = 20190316
2020-06-02 10:16:57|MAIN|INFO|PyAFF4 version = 0.31
2020-06-02 10:16:57|MAIN|INFO|Opened image IMAGE.E01
2020-06-02 10:16:57|MAIN|DEBUG|Skipping EFI System Partition @ offset 20480
2020-06-02 10:16:57|MAIN|INFO|Looking at FS with volume label 'Customer'  @ offset 209735680
2020-06-02 10:16:57|MAIN|INFO|Found an APFS container with uuid: 58E18E64-2D21-442D-BA0C-21EA4F6D60BE
2020-06-02 10:16:57|MAIN.HELPERS.APFS_READER|DEBUG|self.is_sw_encrypted = False
2020-06-02 10:16:57|MAIN.HELPERS.APFS_READER|DEBUG|There are 4 volumes in this container
2020-06-02 10:16:57|MAIN.HELPERS.APFS_READER|DEBUG|Volume Block IDs: [1027, 1030, 154008, 195483], Mapping-omap: 2681122
2020-06-02 10:16:57|MAIN.HELPERS.APFS_READER|DEBUG|Volume Blocks:{1027: 1407544, 1030: 1646153, 154008: 1476348, 195483: 1382481}
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG| -- Volume information:
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  Vol name  = Macintosh HD
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  Num files = 1327877
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  Num dirs  = 294609
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  Vol used  = 821.10 GiB
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  incompatible_features=0x3, fs_flags=0x1
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|root_block_num = 2671243
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG| -- Volume information:
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  Vol name  = Preboot
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  Num files = 60
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  Num dirs  = 17
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  Vol used  = 20.87 MiB
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  incompatible_features=0x1, fs_flags=0x1
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|root_block_num = 2662063
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG| -- Volume information:
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  Vol name  = Recovery
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  Num files = 17
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  Num dirs  = 2
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  Vol used  = 490.60 MiB
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  incompatible_features=0x1, fs_flags=0x1
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|root_block_num = 2668965
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG| -- Volume information:
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  Vol name  = VM
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  Num files = 2
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  Num dirs  = 0
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  Vol used  = 5.00 GiB
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|  incompatible_features=0x1, fs_flags=0x1
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|root_block_num = 2669615
2020-06-02 10:16:58|MAIN.HELPERS.WRITER|ERROR|Query execution error, query was - SELECT Version FROM "Version_Info"
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|ERROR|Error querying volume info from db: no such table: Version_Info
2020-06-02 10:16:58|MAIN|INFO|Found an existing APFS_Volumes.db in the output folder, but it is STALE, creating a new one!
2020-06-02 10:16:58|MAIN|INFO|Reading APFS volumes from container, this may take a few minutes ...
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|0 rows deleted
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|0 rows deleted
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|0 rows deleted
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|0 rows deleted
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|0 rows deleted
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|0 rows deleted
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|DEBUG|0 rows deleted
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|INFO|Vol_2_Preboot Type=inode  Count=79
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|INFO|Vol_2_Preboot Type=xattr  Count=7
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|INFO|Vol_2_Preboot Type=dstream_id  Count=60
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|INFO|Vol_2_Preboot Type=file_extent  Count=64
2020-06-02 10:16:58|MAIN.HELPERS.APFS_READER|INFO|Vol_2_Preboot Type=dir_rec  Count=79
2020-06-02 10:16:59|MAIN.HELPERS.APFS_READER|WARNING|Block values are deleted? ,block=2092400
<snip>
2020-06-02 11:17:38|MAIN.HELPERS.APFS_READER|WARNING|Block values are deleted? ,block=1380069
2020-06-02 11:23:21|MAIN|ERROR|Error while reading APFS volumes
Traceback (most recent call last):
  File "mac_apt_compiled.py", line 345, in FindMacOsPartitionInApfsContainer
  File "plugins\helpers\macinfo.py", line 1087, in ReadApfsVolumes
  File "plugins\helpers\apfs_reader.py", line 420, in read_volume_records
  File "plugins\helpers\apfs_reader.py", line 452, in create_other_tables_and_indexes
  File "plugins\helpers\apfs_reader.py", line 379, in populate_compressed_files_table
  File "plugins\helpers\apfs_reader.py", line 683, in get_raw_decrypted_block
  File "plugins\helpers\apfs_reader.py", line 1391, in get_block
TypeError: unsupported operand type(s) for *: 'NoneType' and 'int'
2020-06-02 11:23:22|MAIN.DISK_REPORT|INFO|Disk info
2020-06-02 11:23:22|MAIN.DISK_REPORT|INFO|Disk Size   = 931.51 GB (1000204886016 bytes)
2020-06-02 11:23:22|MAIN.DISK_REPORT|INFO|Part Scheme = GPT
2020-06-02 11:23:22|MAIN.DISK_REPORT|INFO|Block size  = 512 bytes
2020-06-02 11:23:22|MAIN.DISK_REPORT|INFO|Num Sectors = 1953525168.0 
2020-06-02 11:23:22|MAIN.HELPERS.WRITER|DEBUG|Trying to write out disk, partition & volume information
2020-06-02 11:23:22|MAIN|WARNING|:( Could not find a partition having a macOS installation on it
2020-06-02 11:23:22|MAIN|INFO|--------------------------------------------------
2020-06-02 11:23:22|MAIN|INFO|Finished in time = 01:06:24
2020-06-02 11:23:22|MAIN|INFO|Review the Log file and report any ERRORs or EXCEPTIONS to the developers

Several errors occurred in a few plugins

When I execute mac_apt_mounted_sys_data.py (which was cloned on Feb. 3, 2021), QUICKLOOK plugin and SAVEDSTATE plugin.

QUICKLOOK plugin error:

2021-02-04 15:51:20|MAIN|INFO|Running plugin QUICKLOOK
2021-02-04 15:51:20|MAIN.QUICKLOOK|INFO|QuickLook Cache data found!
2021-02-04 15:51:20|MAIN|ERROR|An exception occurred while running plugin - QUICKLOOK
Traceback (most recent call last):
File "./mac_apt_mounted_sys_data.py", line 217, in
plugin.Plugin_Start(mac_info)
File "/Users/macforensics/Documents/GitHub/mac_apt/plugins/quicklook.py", line 472, in Plugin_Start
parseDbNew(c, quicklook_array, quicklook_db_path, thumbnail_file, mac_info, user)
File "/Users/macforensics/Documents/GitHub/mac_apt/plugins/quicklook.py", line 372, in parseDbNew
apfs_c = export.apfs_db.conn.cursor()
AttributeError: 'MountedMacInfoSeperateSysData' object has no attribute 'apfs_db'

SAVEDSTATE plugin error:

2021-02-04 15:51:20|MAIN|INFO|Running plugin SAVEDSTATE
2021-02-04 15:51:20|MAIN.HELPERS.MACINFO|ERROR|Exception in _GetFileSizeNoPathMod() : [Errno 2] No such file or directory: '/Volumes/Macintosh HD - Data/Users/macforensics/Library/Saved Application State/com.apple.calculator.savedState'
2021-02-04 15:51:20|MAIN.HELPERS.MACINFO|ERROR|Exception in _GetFileSizeNoPathMod() : [Errno 2] No such file or directory: '/Volumes/Macintosh HD - Data/Users/macforensics/Library/Saved Application State/com.apple.AppStore.savedState'
2021-02-04 15:51:20|MAIN.HELPERS.MACINFO|ERROR|Exception in _GetFileSizeNoPathMod() : [Errno 2] No such file or directory: '/Volumes/Macintosh HD - Data/Users/macforensics/Library/Saved Application State/com.apple.iMovieApp.savedState'
2021-02-04 15:51:20|MAIN.HELPERS.MACINFO|ERROR|Exception in _GetFileSizeNoPathMod() : [Errno 2] No such file or directory: '/Volumes/Macintosh HD - Data/Users/macforensics/Library/Saved Application State/com.skitch.skitch.savedState'

These .savedState files above are symlink files, but their target files don't exist.
Check .savedState file:

(mac_apt) macforensics@ermacforensics mac_apt % ls -al /Volumes/Macintosh\ HD\ -\ Data/Users/macforensics/Library/Saved\ Application\ State
(snip)
lrwxr-xr-x   1 macforensics  staff   128  9  5  2019 com.apple.calculator.savedState -> /Users/macforensics/Library/Containers/com.apple.calculator/Data/Library/Saved Application State/com.apple.calculator.savedState
(snip)

Check symlink target file (It doesn't exist):

(mac_apt) macforensics@ermacforensics mac_apt % ls -al /Users/macforensics/Library/Containers/com.apple.calculator/Data/Library/Saved\ Application\ State/com.apple.calculator.savedState
ls: /Users/macforensics/Library/Containers/com.apple.calculator/Data/Library/Saved Application State/com.apple.calculator.savedState: No such file or directory

Finally, an exception occurred like below.

Exception ignored in: <function SqliteWriter.del at 0x7f9d781657b8>
Traceback (most recent call last):
File "/Users/macforensics/Documents/GitHub/mac_apt/plugins/helpers/writer.py", line 402, in del
ValueError: SqliteWriter destructor, Dear coder, you forgot to close db.

I have attached a log file too.
Log.20210204-155117.txt

TypeError occurs while reading APFS volumes

I've seen a TypeError in RecurseReadTree() like below, when I try to analyze a disk image (AFF4) that created with Cellebrite Digital Collector (former MacQuisition).

2021-09-01 16:55:38|MAIN|ERROR|Error while reading APFS volumes
Traceback (most recent call last):
  File "/Users/macforensics/Documents/GitHub/mac_apt/./mac_apt.py", line 307, in FindMacOsPartitionInApfsContainer
    mac_info.ReadApfsVolumes()
  File "/Users/macforensics/Documents/GitHub/mac_apt/plugins/helpers/macinfo.py", line 1079, in ReadApfsVolumes
    apfs_parser.read_volume_records()
  File "/Users/macforensics/Documents/GitHub/mac_apt/plugins/helpers/apfs_reader.py", line 424, in read_volume_records
    self.RecurseReadTree(self.volume.root_block_num, root_block, my_root, False, self.volume.root_tree_oid, 0, 0)
  File "/Users/macforensics/Documents/GitHub/mac_apt/plugins/helpers/apfs_reader.py", line 527, in RecurseReadTree
    self.RecurseReadTree(new_block_num, newblock, my_node, force_fstree, root_oid, oid, xid, debug_parent_list)
  File "/Users/macforensics/Documents/GitHub/mac_apt/plugins/helpers/apfs_reader.py", line 527, in RecurseReadTree
    self.RecurseReadTree(new_block_num, newblock, my_node, force_fstree, root_oid, oid, xid, debug_parent_list)
  File "/Users/macforensics/Documents/GitHub/mac_apt/plugins/helpers/apfs_reader.py", line 527, in RecurseReadTree
    self.RecurseReadTree(new_block_num, newblock, my_node, force_fstree, root_oid, oid, xid, debug_parent_list)
  File "/Users/macforensics/Documents/GitHub/mac_apt/plugins/helpers/apfs_reader.py", line 535, in RecurseReadTree
    location_oid = entry.data + (root_oid if no_blk_hdr_force_subtype_fstree else 0)
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'
2021-09-01 16:55:38|MAIN|ERROR|Error while trying to read partitions on disk
Traceback (most recent call last):
  File "/Users/macforensics/Documents/GitHub/mac_apt/./mac_apt.py", line 572, in <module>
    vol_info = pytsk3.Volume_Info(img)
OSError: Volume_Info_Con: (tsk3.c:662) Error opening Volume_Info: Cannot determine partition type

Also, attaching a log file.
Log.20210901-165503.txt

parser PRINTJOBS: error

When executing the PRINTJOBS-parser, it stumbles upon a value which is not an UNIX-timestamp.

The script errors and exits.

This can be bypassed the dirty way in common.py by adding the following under ReadUnixTime

                elif isinstance(unix_time, bytes) and len(unix_time) == 0:
                    unix_time = 0

Optionally, a error-warning can be written also.

[Help!! Urgently] Export notes data from macbook pro

Hi Guys

My computer is macbook pro, I got issue with my drive, but luckly, I got backup below files.

May please let me know how can restore my data? thanks

QuarantineEvents database

This may not be an issue with this awesome tool ... but the way apple stores artifacts/events. I dont see QE contain the files downloaded using apple mail.app. Am I missing something ? Appreciate any clue. Thank you

OS: darwin 10.15.7

PS: Apple support page HT201940 (archived now) shows mail.app is quarantine aware

plugins/helpers/writer.py", line 643, in __del__ ValueError: ExcelWriter destructor, Dear coder, you forgot to close file.

I was trying recover auto-sync-deleted iCloud notes from my MacBook Pro(Mojave).
Follow the tutorial to install on my mac:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
sudo chown -R $(whoami) /usr/local/bin /usr/local/lib /usr/local/sbin
chmod u+w /usr/local/bin /usr/local/lib /usr/local/sbin
brew install python3 git
sudo pip3 install --upgrade virtualenv
git clone --recursive https://github.com/ydkhatri/pylzfse
cd pylzfse
python setup.py build
python setup.py install
pip3 install pytsk3==20170802 libvmdk-python==20181227

tar xfz ./libewf-20140808.tar.gz
cd libewf-20140808
python setup.py build
python setup.py install
pip install biplist construct==2.9.45 xlsxwriter plistutils kaitaistruct lz4 pycryptodome cryptography pybindgen==0.21.0 pillow
cd ..
sudo pip3 install pyaff4-0.31-yk.zip
git clone https://github.com/libyal/libewf-legacy/
cd libewf-20140808
python3 setup.py build
sudo python3 setup.py install
cd
cd mac_apt/
source env/bin/activate

(env) jianw-mbp:mac_apt jianw$ cp /Users/jianw/Library/Group\ Containers/group.com.apple.notes/NoteStore.sqlite ~/Desktop/

(env) jianw-mbp:mac_apt jianw$ python ./mac_apt.py -x -o output E01 /Users/jianw/Desktop/NoteStore.sqlite NOTES -l DEBUG
Output path was : /Users/jianw/mac_apt/output
MAIN-INFO-Started macOS Artifact Parsing Tool, version 0.6
MAIN-INFO-Dates and times are in UTC unless the specific artifact being parsed saves it as local time!
MAIN-DEBUG-./mac_apt.py -x -o output E01 /Users/jianw/Desktop/NoteStore.sqlite NOTES -l DEBUG
MAIN-INFO-Pytsk version = 20170801
MAIN-INFO-Pyewf version = 20140808
MAIN-INFO-Pyvmdk version = 20181227
MAIN-INFO-PyAFF4 version = 0.31
MAIN-ERROR-Failed to load image. Error Details are: pyewf_glob: unable to glob filenames. libewf_glob: invalid filename - missing extension.

Exception ignored in: <function SqliteWriter.del at 0x105a95f80>
Traceback (most recent call last):
File "/Users/jianw/mac_apt/plugins/helpers/writer.py", line 402, in del
ValueError: SqliteWriter destructor, Dear coder, you forgot to close db.
Exception ignored in: <function ExcelWriter.del at 0x105a98b00>
Traceback (most recent call last):
File "/Users/jianw/mac_apt/plugins/helpers/writer.py", line 643, in del
ValueError: ExcelWriter destructor, Dear coder, you forgot to close file.

(env) jianw-mbp:mac_apt jianw$ ls -lrt ./output
total 8
drwxr-xr-x 3 jianw staff 96 Jun 1 20:06 Export
-rw-r--r-- 1 jianw staff 0 Jun 1 20:06 mac_apt.db
-rw-r--r-- 1 jianw staff 697 Jun 1 20:06 Log.20200601-200610.txt

I rename the sqlite file to txt in order to upload.
NoteStore.sqlite.txt

Error after source code python 3 install

I get this after running the install sh and starting the env and run "python mac_apt.py -h"

Traceback (most recent call last):
File "mac_apt.py", line 23, in
import plugins.helpers.macinfo as macinfo
File "/Users/examiner/Desktop/mac_apt/mac_apt/plugins/helpers/macinfo.py", line 26, in
from plugins.helpers.hfs_alt import HFSVolume
File "/Users/examiner/Desktop/mac_apt/mac_apt/plugins/helpers/hfs_alt.py", line 29, in
from plugins.helpers.btree import AttributesTree, CatalogTree, ExtentsOverflowTree
File "/Users/examiner/Desktop/mac_apt/mac_apt/plugins/helpers/btree.py", line 18, in
from plugins.helpers.structs import *
File "/Users/examiner/Desktop/mac_apt/mac_apt/plugins/helpers/structs.py", line 185, in
"unicode" / PaddedString(lambda ctx: ctx["length"] * 2, encoding="utf-16-be") # "unicode",
NameError: name 'PaddedString' is not defined

macOS Installation Script issue with virtualenv

Hello Yogesh,
while using the macOS Installation Script "mac_aptInstall-macOS-v21.sh" I am getting an error: "The folder in Line 118 does not exist" and mac_apt won't install properly. It seems to be an issue dealing with virtualenv. Changing line 112 from "virtualenv --python python3 env &> /dev/null" to "python3 -m venv env &> /dev/null" did solve this problem. I am using the latest macOS 11.6, with Python 3.7.0 and latest hombrew updates. Maybe I am not the only one ;-)

Kind Regards
Stefan
my4n6

ios_apt: NameError: name 'ai' is not defined

When executing ios_apt on a GK-dump, I get the following error:

Traceback (most recent call last):
  File "C:\github\mac_apt\ios_apt_compiled.py", line 191, in <module>
  File "plugins\helpers\macinfo.py", line 2106, in _GetAppDetails
NameError: name 'ai' is not defined
[19336] Failed to execute script ios_apt_compiled
MAIN.HELPERS.MACINFO-DEBUG-Exception while deleting temp files/folders: [WinError 32] The process cannot access the file because it is being used by another process: './TempNBJQ\\applicationState.db'

The complete DEBUG log is send to you by e-mail.
Thank you

Notes plugin loses CR, LF, and tab when exporting to csv file

When exporting notes to a csv file, CR, LF, and tab are all replaced by a comma
to avoid corrupting the format of the csv file. A better approach would be to
convert these characters into some sequence that could be reversed later.
For example, in plugins/helpers/writer.py, make this change:

--- a/plugins/helpers/writer.py
+++ b/plugins/helpers/writer.py
@@ -381,7 +381,7 @@ class CsvWriter:
         for item in row:
             safe_str = unicode(item)
             try:
-                safe_str = safe_str.replace('\r\n', ',').replace('\r', ',').replace('\n', ',').replace('\t', ' ')
+                safe_str = safe_str.replace('\\', '\\\\').replace('\r', '\\r').replace('\n', '\\n').replace('\t', '\\t')
             except Exception as ex:
                 log.exception()
             safe_list.append(safe_str)

For compatibility, you might want to introduce this change with a command line switch.

Running on Ubuntu 64bit 16.04

Hello! I see you tested it on 32bit Ubuntu. On my version, everything installed correctly, but upon running it I get:

root@ubuntu:/usr/local/bin/mac_apt-0.2.6-alpha# python mac_apt.py
Traceback (most recent call last):
File "mac_apt.py", line 29, in
import plugins.helpers.macinfo as macinfo
File "/usr/local/bin/mac_apt-0.2.6-alpha/plugins/helpers/macinfo.py", line 24, in
from hfs_alt import HFSVolume
File "/usr/local/bin/mac_apt-0.2.6-alpha/plugins/helpers/hfs_alt.py", line 30, in
from btree import AttributesTree, CatalogTree, ExtentsOverflowTree
File "/usr/local/bin/mac_apt-0.2.6-alpha/plugins/helpers/btree.py", line 19, in
from structs import *
File "/usr/local/bin/mac_apt-0.2.6-alpha/plugins/helpers/structs.py", line 85, in
"startBlock" / Int32ub,
NameError: name 'Int32ub' is not defined

This is the excerpt from the source code of structs.py:

Line 84:     HFSPlusExtentDescriptor = "HFSPlusExtentDescriptor" / Struct(
Line 85:         "startBlock" / Int32ub,
Line 86:         "blockCount" / Int32ub
Line 87:     )

Would you happen to have any ideas on how I would tackle this problem? I have searched for anything I can regarding Int32ub but I have been unsuccessful. I'm assuming maybe it has something to do with 32bit but I can't be sure.

Thanks,
Clint Baker

com.apple.Safari.plist under Containers folder is not analyzed

The latest mac_apt (1.3.1) does not analyze com.apple.Safari.plist under Containers folder (/Users/*/Library/Containers/com.apple.Safari/Data/Library/Preferences/com.apple.Safari.plist).

In addition, it is missing both AppExtensions (/Users/*/Library/Containers/com.apple.Safari/Data/Library/Safari/AppExtensions/Extensions.plist) and WebExtensions (/Users/*/Library/Containers/com.apple.Safari/Data/Library/Safari/WebExtensions/Extensions.plist).

Problem with execution

Thank you for making available such a great parser.

I installed the latest version of mac_apt as per your installation instructions but I am getting errors and can't get any results. I used Python 7 as well as Python 8, updated the construct to version 2.10 and the errors are the same. Please see below:

C:\Users\TinoK>python C:\mac_apt\mac_apt.py -h
Traceback (most recent call last):
File "C:\mac_apt\mac_apt.py", line 23, in
import plugins.helpers.macinfo as macinfo
File "C:\mac_apt\plugins\helpers\macinfo.py", line 11, in
import biplist
ModuleNotFoundError: No module named 'biplist'

Any suggestions please?

Thank you
Tino K.

MAIN-ERROR-Failed to load image.

python3
macOS Mojave 10.14.6

what dose input_path means ? where should l download it or find it locally?

Please add requirements.txt

While running mac_apt.py from git clone
ImportError: No module named kaitaistruct

Please add a requirements.txt file that contains information about dependencies.

only reference I found regarding "dependencies" was

mac_apt/plugins/helpers/spotlight_parser.py

Line 22 in 2730231

# Dependencies can be installed using the command 'pip install lz4 enum34'

Installation pyaff4 - update wiki guide

Working on Windows 10 with Python 3.9 (64bit)

When following your guide for installing pyaff4, one requirement, namely fastchunking, fails to install with pip, causing pyaff4 not to be installed.

The solution for me was:

• downloading it here: https://pypi.org/project/fastchunking/#files
• unzipping
• python setup.py build and python setup.py install
• finally pip install pyaff4-0.31-yk.zip

consolidated.db may not always provide the correct serial number

I am parsing a HFS image with mac_apt v0.5
As expected, it finds and reads private\var\folders\zz\zyxvpxvq6csfxvn_n00000sm00006d\C\consolidated.db
But the found serial number in that db, is not the one we found engraved on the Mac itself.

After further inspection, we found an earlier version of consolidated.db in the deleted files. We were able to correctly recover this file.

This db contains the correct serial, namely as found on the device itself.

How can we explain this?
Now we know this, is it still safe to believe consolidated.db ?

I'll attach the two files: consolidated.zip

macOS 10.15 - Catalina compatibility - Need firmlinks implemented

On a catalina image, the main root partition is split into Macintosh HD and Macintosh HD - Data. It uses firmlinks defined in /usr/share/firmlink to merge the volumes at the given locations to present it as a single view to the user.

mac_apt currently does not handle firmlinks, and will need some code rewrite for this.

error running precompiled bundles (.app)

Tried running both v0.5 & 0.4 on Mac OS 10.14.6 ... but ended up with a fatal error (python runtime not located)

Appreciate help :)

missing lz4.block

I installed it on ubuntu 18.04 having latest sans sift V3. Followed installation guide https://github.com/ydkhatri/mac_apt/wiki/Installation

I see this error .. any clue ? Thanks

python mac_apt.py -h
!!Error in plugin 'spotlight.py' at line 32 - No module named lz4.block

parse specific data copied on the disk

Is it possible to parse specific artifacts ? for ex: browser/chrome data, I collected using kape from a windows system.

I noticed that data collected using AXIOM can be parsed. What input_type should be supplied ? Thanks

python3 mac_apt.py -o /output_dir -c  input_type /input_dir  CHROME

Allow bulk analysis of UTMPX files

I think it'd be beneficial to add capability to bulk analyze UTMPX files when given a folder instead of a filepath.

I can probably implement this when I get back home in a couple weeks is this is a direction that is agreed on, just needs to tweak the output to not keep generating new files per each inputted UTMPX file, and small logic for file vs folder.

Also hi Yogesh :)

ReadLastGKRejectPlist() function in QUARANTINE plugin does not store its results to DB

The results of ReadLastGKRejectPlist() function are displayed with log.info(), but not stored in mac_apt.db.
Is this behavior a bug or not?

tons of "item_type unknown (0xF2)" and "Unknown custom data object type" occur while processing Unified Logs

The warning and info messages like below are displayed, if I analyze unified logs of macOS 10.6.

2021-09-29 11:29:09|MAIN.UNIFIED_LOG_READER_LIB|INFO|Unknown custom data object type '{public,mdns:dnshdr}' data size=0xC in log @ 0x18E8
2021-09-29 11:29:09|MAIN.UNIFIED_LOG_READER_LIB|INFO|Unknown custom data object type '{public,mdns:dnshdr}' data size=0xC in log @ 0x1988
2021-09-29 11:29:09|MAIN.UNIFIED_LOG_READER_LIB|INFO|Unknown custom data object type '{public,mdns:dnshdr}' data size=0xC in log @ 0x1A28
2021-09-29 11:29:09|MAIN.UNIFIED_LOG_READER_LIB|INFO|Unknown custom data object type '{public,mdns:dnshdr}' data size=0xC in log @ 0x1B68
2021-09-29 11:29:09|MAIN.UNIFIED_LOG_READER_LIB|WARNING|item_type unknown (0xF2)
2021-09-29 11:29:09|MAIN.UNIFIED_LOG_READER_LIB|WARNING|item_type unknown (0xF2)
2021-09-29 11:29:09|MAIN.UNIFIED_LOG_READER_LIB|WARNING|item_type unknown (0xF2)
2021-09-29 11:29:09|MAIN.UNIFIED_LOG_READER_LIB|INFO|Unknown custom data object type '{private, mask.hash, mdnsresponder:domain_name}' data size=0x4 in log @ 0x1E20
2021-09-29 11:29:09|MAIN.UNIFIED_LOG_READER_LIB|WARNING|item_type unknown (0xF2)
2021-09-29 11:29:09|MAIN.UNIFIED_LOG_READER_LIB|WARNING|item_type unknown (0xF2)

A very large number of messages have been recorded.

% fgrep "MAIN.UNIFIED_LOG_READER_LIB" Log.20210929-105600.txt | fgrep "item_type unknown (0xF2)" | wc -l               
 1365407
% fgrep "MAIN.UNIFIED_LOG_READER_LIB" Log.20210929-105600.txt | fgrep "Unknown custom data object type" | wc -l
  889319

mac_apt log file has been attached.
Log.20210929-105600.txt.zip

Also, I can share the unified log files exported by mac_apt if necessary to fix the problem.

BASHSESSION not pulling all sessions

Testing in macOS 10.14.4 - only .bash_history and 2 sessions parsed on image with 4 sessions.

Spotlight parser error

I've seen another spotlight plugin's parser error like below:

2021-09-05 21:47:02|MAIN.SPOTLIGHT.SPOTLIGHT_PARSER|ERROR|Error trying to process item @ block 9D014 offset 0
Traceback (most recent call last):
  File "/Users/macforensics/Documents/GitHub/forked/mac_apt/plugins/helpers/spotlight_parser.py", line 830, in ParseMetadataBlocks
    md_item.ParseItem(self.properties, self.categories, self.indexes_1, self.indexes_2)
  File "/Users/macforensics/Documents/GitHub/forked/mac_apt/plugins/helpers/spotlight_parser.py", line 292, in ParseItem
    singles = [self.ReadSingleByte() for x in range(num_values)]
  File "/Users/macforensics/Documents/GitHub/forked/mac_apt/plugins/helpers/spotlight_parser.py", line 292, in <listcomp>
    singles = [self.ReadSingleByte() for x in range(num_values)]
  File "/Users/macforensics/Documents/GitHub/forked/mac_apt/plugins/helpers/spotlight_parser.py", line 151, in ReadSingleByte
    single = struct.unpack("<B", self.data[self.pos : self.pos + 1])[0]
struct.error: unpack requires a buffer of 1 bytes
2021-09-05 21:47:21|MAIN.SPOTLIGHT.SPOTLIGHT_PARSER|ERROR|Error, cannot proceed, invalid property index 331, skip=68
2021-09-05 21:47:21|MAIN.SPOTLIGHT.SPOTLIGHT_PARSER|ERROR|Error trying to process item @ block 2285014 offset 17300
Traceback (most recent call last):
  File "/Users/macforensics/Documents/GitHub/forked/mac_apt/plugins/helpers/spotlight_parser.py", line 830, in ParseMetadataBlocks
    md_item.ParseItem(self.properties, self.categories, self.indexes_1, self.indexes_2)
  File "/Users/macforensics/Documents/GitHub/forked/mac_apt/plugins/helpers/spotlight_parser.py", line 327, in ParseItem
    value = self.ReadDate()
  File "/Users/macforensics/Documents/GitHub/forked/mac_apt/plugins/helpers/spotlight_parser.py", line 102, in ReadDate
    mac_abs_time = struct.unpack("<q", struct.pack("<Q", int(mac_abs_time)) )[0] # double to signed int64
struct.error: argument out of range

The archive file is AFF4 that was created with Cellebrite Digital Collector.
Also, attaching the log file.
Log.20210905-213434.txt

I can also provide the spotlight files for you.

mac_info.IsValidFolderPath() always returns False?

Has the behavior of mac_info.IsValidFolderPath() changed?
IsValidFolderPath() always returns False whether I give it the correct path that exists in the disk image or the wrong path.
As a result, the CFURLCACHE and CHROME plugins will not work properly.

I have modified CFURLCACHE plugin to test it like below:

The result:

MAIN.CFURLCACHE-INFO-/Users/macforensics/Library/Caches/ is NOT valid.
MAIN.CFURLCACHE-INFO-/Users/macforensics/Library/Caches/ZZZZ is NOT valid.
MAIN.CFURLCACHE-INFO-/Users/Guest/Library/Caches/ is NOT valid.
MAIN.CFURLCACHE-INFO-/Users/Guest/Library/Caches/ZZZZ is NOT valid.

I confirmed that cache folders are existed.

MAIN-ERROR-Failed to load image. Error Details are: pyewf_glob: unable to free globbed filenames. libewf_glob_free: invalid filenames.

I am trying to recover my corrupted notes on Mac OS Catalina.

Using the following command -

python3 mac_apt.py -x -o ~/Case_Output E01 ~/Acquisition.E01 NOTES

Please know that I do not understand how to specify the input file. I am simply using the sample. I am not sure what Acquistion.E01 means. Can I run this plugin standalone? I do have the NoteStore.sqlite file handy.

Getting following output -

Output path was : /Users/priyank/Case_Output
MAIN-INFO-Started macOS Artifact Parsing Tool, version 1.3.1
MAIN-INFO-Dates and times are in UTC unless the specific artifact being parsed saves it as local time!
MAIN-INFO-Python version = 3.9.5 (v3.9.5:0a7dcbdb13, May 3 2021, 13:17:02)
[Clang 6.0 (clang-600.0.57)]
MAIN-INFO-Pytsk version = 20210419
MAIN-INFO-Pyewf version = 20201230
MAIN-INFO-Pyvmdk version = 20210418
MAIN-INFO-PyAFF4 version = 0.33
MAIN-ERROR-Failed to load image. Error Details are: pyewf_glob: unable to free globbed filenames. libewf_glob_free: invalid filenames.

Exception ignored in: <function SqliteWriter.del at 0x10b8a80d0>
Traceback (most recent call last):
File "/Users/priyank/Downloads/mac_apt-1.3.1/plugins/helpers/writer.py", line 402, in del
ValueError: SqliteWriter destructor, Dear coder, you forgot to close db.
Exception ignored in: <function ExcelWriter.del at 0x10b8a8c10>
Traceback (most recent call last):
File "/Users/priyank/Downloads/mac_apt-1.3.1/plugins/helpers/writer.py", line 643, in del
ValueError: ExcelWriter destructor, Dear coder, you forgot to close file.

How to handle storedata files?

I just lost all my notes accidentally and i tried to recover them using this tool. Unfortunately i wasn't able to make it work. I tried to run

python ~/Desktop/NOTES/apps/mac_apt-master/mac_apt_artifact_only.py NOTES -o ~/output -i ~/Desktop/NOTES/data/NotesV7.storedata

without any luck:

2021-01-24 17:39:26|MAIN|INFO|Started macOS Artifact Parsing Tool - Artifact Only mode, version 0.9.dev
2021-01-24 17:39:26|MAIN|INFO|Dates and times are in UTC unless the specific artifact being parsed saves it as local time!
2021-01-24 17:39:26|MAIN|INFO|--------------------------------------------------
2021-01-24 17:39:26|MAIN|INFO|Running plugin NOTES
2021-01-24 17:39:26|MAIN|INFO|--------------------------------------------------
2021-01-24 17:39:26|MAIN.NOTES|INFO|Module Started as standalone
2021-01-24 17:39:26|MAIN.NOTES|INFO|Processing file /Users/iwanuschka/Desktop/NOTES/daten/NotesV7.storedata
2021-01-24 17:39:26|MAIN.NOTES|ERROR|Failed to open database, is it a valid Notes DB?
Traceback (most recent call last):
  File "/Users/iwanuschka/Downloads/mac_apt-master/plugins/notes.py", line 376, in OpenDb
    conn = CommonFunctions.open_sqlite_db_readonly(inputPath)
  File "/Users/iwanuschka/Downloads/mac_apt-master/plugins/helpers/common.py", line 179, in open_sqlite_db_readonly
    return sqlite3.connect (f"file:{path}?mode=ro", uri=True)
sqlite3.OperationalError: unable to open database file
2021-01-24 17:39:26|MAIN.NOTES|INFO|No notes found in /Users/iwanuschka/Desktop/NOTES/daten/NotesV7.storedata
2021-01-24 17:39:26|MAIN|INFO|--------------------------------------------------
2021-01-24 17:39:26|MAIN|INFO|Finished in time = 00:00:00
2021-01-24 17:39:26|MAIN|INFO|Review the Log file and report any ERRORs or EXCEPTIONS to the developers

I went through the installation process described in https://drive.google.com/uc?export=download&id=1JEYlq_3WX6DgaLgm_LZ6jzPRT4iEg5Ge. The only exception: i am using pyenv (=> pyenv shell 3.7.9) instead of python3.

NotesV7.storedata has been taken from a system running Mac OS Catalina.

Do you have any idea how to solve this problem?

How could I get one memory dump on one macbookpro?

I search a lot, but no idea

binary distribution of lzfse is missing license details

If lzfse.pyd is compiled from: https://github.com/lzfse/lzfse

From: https://github.com/lzfse/lzfse/blob/master/LICENSE

2.  Redistributions in binary form must reproduce the above copyright notice,
    this list of conditions and the following disclaimer in the documentation
    and/or other materials provided with the distribution.

Please add some information about the origin, build parameters (32-bit or 64-bit, compiler used) of this binary file.

Error when using WIFI plugin

I am using release v1.3.1.

When I try to parse an APFS image with WIFI-plugin, I get the following error:

MAIN-ERROR-An exception occurred while running plugin - WIFI
Traceback (most recent call last):
  File "C:\Github\mac_apt\mac_apt_compiled.py", line 671, in <module>
  File "plugins\airport_preferences.py", line 334, in Plugin_Start
  File "plugins\airport_preferences.py", line 268, in ParseWifiFromImage
  File "plugins\airport_preferences.py", line 289, in ReadKnownNetworksPlist
TypeError: '>' not supported between instances of 'datetime.datetime' and 'str'

For that reason, the output for WIFI stays empty.
I sent you the detailed DEBUG-log by e-mail.

vmdk Mac Images

Hello, i tested the tool with an vmdk image and it doesn't work , does anyone know any way to make it work? or to convert it to an accepted format?

Parsing public.url attachments in note BLOB

Currently, mac_apt does not extract public.url attachments in the note BLOB of Apple Notes.

When creating a note from a web page, a note containing a public.url attachment is created.

Since the Notes app allows the user to choose which note to use, a note may contain multiple public.url attachments.

Each public.url attachment in the note BLOB contains an UUID that can be used to look up the URL.

This will only extract one public.url attachment from the note BLOB:

e.g. High Sierra

SELECT n.Z_PK, n.ZNOTE as note_id, n.ZDATA as data, 
c3.ZFILESIZE, 
c4.ZFILENAME, c4.ZIDENTIFIER as att_uuid, c3.ZURLSTRING as att_url, 
c1.ZTITLE1 as title, c1.ZSNIPPET as snippet, c1.ZIDENTIFIER as noteID, 
c1.ZCREATIONDATE1 as created, c1.ZLASTVIEWEDMODIFICATIONDATE, c1.ZMODIFICATIONDATE1 as modified, 
c2.ZACCOUNT3, c2.ZTITLE2 as folderName, c2.ZIDENTIFIER as folderID, 
c5.ZNAME as acc_name, c5.ZIDENTIFIER as acc_identifier, c5.ZACCOUNTTYPE 
FROM ZICNOTEDATA as n 
LEFT JOIN ZICCLOUDSYNCINGOBJECT as c1 ON c1.ZNOTEDATA = n.Z_PK 
LEFT JOIN ZICCLOUDSYNCINGOBJECT as c2 ON c2.Z_PK = c1.ZFOLDER 
LEFT JOIN ZICCLOUDSYNCINGOBJECT as c3 ON c3.ZNOTE= n.ZNOTE 
LEFT JOIN ZICCLOUDSYNCINGOBJECT as c4 ON c4.ZATTACHMENT1= c3.Z_PK 
LEFT JOIN ZICCLOUDSYNCINGOBJECT as c5 ON c5.Z_PK = c1.ZACCOUNT2 
ORDER BY note_id

mac_apt_artifact_only tries to use nonexistent python by default

In the current release, running./mac_apt_artifact_only -h after initial download gives the following output:

2020-04-23 14:08:53.828 mac_apt_artifact_only[43079:479791] A Python runtime not could be located.  You may need to install a framework build of Python, or edit the PyRuntimeLocations array in this application's Info.plist file.

Examining the Info.plist shows that the app is trying to find a version of python at ../Frameworks/Python.framework/Versions/3.7/Python, but Versions/3.7 is empty. However, Versions/Current contains a Python file that allows the program to run successfully.

Accordingly, the package should be released with the appropriate path in Info.plist in the first place (since the Python provided is part of the package) instead of forcing users to figure this out.

export of USERS

When specifying only one parser (no matter which one), there is - as expected - an export of the files which are examined.

However, there is always an extra export of the USERS files (plist). This is not expected because the USERS parser was not specified in the command.
(The export of the user.plist files is also logged in the Exported_Files_Log.csv)

Notes plugin fails on High Sierra

The script relies on retrieving information from a table that does not exist on the HighSierra (macOS 10.13) version of Notes database.

From the output Log:

2018-01-24 12:09:27|MAIN.HELPERS.MACINFO|DEBUG|Trying to extract and read db: /Users/johnsmith/Library/Group Containers/group.com.apple.notes/NoteStore.sqlite
2018-01-24 12:09:27|MAIN.NOTES|DEBUG|Opened database successfully
2018-01-24 12:09:27|MAIN.NOTES|ERROR|Query  execution failed. Query was:  SELECT n.Z_12FOLDERS as folder_id , n.Z_9NOTES as note_id, d.ZDATA as data,  c2.ZTITLE2 as folder, c2.ZDATEFORLASTTITLEMODIFICATION as folder_title_modified,  c1.ZCREATIONDATE as created, c1.ZMODIFICATIONDATE1 as modified, c1.ZSNIPPET as snippet, c1.ZTITLE1 as title, c1.ZACCOUNT2 as acc_id,  c5.ZACCOUNTTYPE as acc_type, c5.ZIDENTIFIER as acc_identifier, c5.ZNAME as acc_name,  c3.ZMEDIA as media_id, c3.ZFILESIZE as att_filesize, c3.ZMODIFICATIONDATE as att_modified, c3.ZPREVIEWUPDATEDATE as att_previewed, c3.ZTITLE as att_title, c3.ZTYPEUTI, c3.ZIDENTIFIER as att_uuid,  c4.ZFILENAME, c4.ZIDENTIFIER as media_uuid  FROM Z_12NOTES as n  LEFT JOIN ZICNOTEDATA as d ON d.ZNOTE = n.Z_9NOTES  LEFT JOIN ZICCLOUDSYNCINGOBJECT as c1 ON c1.Z_PK = n.Z_9NOTES  LEFT JOIN ZICCLOUDSYNCINGOBJECT as c2 ON c2.Z_PK = n.Z_12FOLDERS  LEFT JOIN ZICCLOUDSYNCINGOBJECT as c3 ON c3.ZNOTE = n.Z_9NOTES  LEFT JOIN ZICCLOUDSYNCINGOBJECT as c4 ON c3.ZMEDIA = c4.Z_PK  LEFT JOIN ZICCLOUDSYNCINGOBJECT as c5 ON c5.Z_PK = c1.ZACCOUNT2  ORDER BY note_id 
Traceback (most recent call last):
  File "c:\Github\mac_apt\plugins\notes.py", line 208, in ReadNotes
    cursor = db.execute(query)
OperationalError: no such table: Z_12NOTES

A1466 image with APFS is not processing

In my case an image with APFS volume cannot be processed by mac_apt. The following error will be seen:

$ python ~/mac_apt/mac_apt.py -o rep DD hard.raw ALL
Output path was : rep
MAIN-INFO-Started macOS Artifact Parsing Tool, version 0.3
MAIN-INFO-Dates and times are in UTC unless the specific artifact being parsed saves it as local time!
MAIN-INFO-Pytsk version = 20170801
MAIN-INFO-Pyewf version = 20140608
MAIN-INFO-Opened image hard.raw
MAIN-INFO-Looking at FS with volume label ''  @ offset 209735680
MAIN-INFO-Found an APFS container with uuid: B1F3F707-ED0B-49A6-B4DB-7439A47BB405
MAIN-INFO-Reading APFS volumes from container, this may take a few minutes ...
MAIN-ERROR-Error while reading APFS volumes
Traceback (most recent call last):
  File "/home/sansforensics/mac_apt/mac_apt.py", line 173, in FindOsxPartitionInApfsContainer
    mac_info.ReadApfsVolumes()
  File "/home/sansforensics/mac_apt/plugins/helpers/macinfo.py", line 844, in ReadApfsVolumes
    apfs_parser.read_volume_records()
  File "/home/sansforensics/mac_apt/plugins/helpers/apfs_reader.py", line 268, in read_volume_records
    self.read_entries(self.volume.root_block_num, root_block)
  File "/home/sansforensics/mac_apt/plugins/helpers/apfs_reader.py", line 352, in read_entries
    self.read_entries(entry.data.pointer, newblock)
  File "/home/sansforensics/mac_apt/plugins/helpers/apfs_reader.py", line 352, in read_entries
    self.read_entries(entry.data.pointer, newblock)
  File "/home/sansforensics/mac_apt/plugins/helpers/apfs_reader.py", line 355, in read_entries
    self.read_entries(entry.data.block_num.value, newblock)
  File "/home/sansforensics/mac_apt/plugins/helpers/apfs_reader.py", line 307, in read_entries
    if type(entry.data) == self.ptr_type: #apfs.Apfs.PointerRecord:
  File "/home/sansforensics/mac_apt/plugins/helpers/apfs.py", line 212, in data
    self._m_data = self._root.ExtattrRecord(self._io, self, self._root)
  File "/home/sansforensics/mac_apt/plugins/helpers/apfs.py", line 456, in __init__
    self.type_ea = self._root.EaType(self._io.read_u2le())
  File "/usr/local/lib/python2.7/dist-packages/enum/__init__.py", line 347, in __call__
    return cls.__new__(cls, value)
  File "/usr/local/lib/python2.7/dist-packages/enum/__init__.py", line 662, in __new__
    raise ValueError("%s is not a valid %s" % (value, cls.__name__))
ValueError: 17 is not a valid EaType
MAIN.DISK_REPORT-INFO-Disk info
MAIN.DISK_REPORT-INFO-Disk Size   = 233.76 GB (251000193024 bytes)
MAIN.DISK_REPORT-INFO-Part Scheme = GPT
MAIN.DISK_REPORT-INFO-Block size  = 512 bytes
MAIN.DISK_REPORT-INFO-Num Sectors = 490234752.0 
MAIN-WARNING-:( Could not find a partition having an OSX installation on it
MAIN-INFO-Finished in time = 00:00:03
MAIN-INFO-Review the Log file and report any ERRORs or EXCEPTIONS to the developers

Feature Request: Zip folder support for ios_apt

I have no idea how difficult it would be but would love to see it, as we see more FFS extractions.

Get file created (birth) time on linux

Currently modified time is passed as created/birth time because of python's limitation (dependency on stat). This issue only affects running mac_apt in MOUNTED mode when the platform is linux.

The pyaff project has this implemented here: https://github.com/aff4/pyaff4/pull/6/files
We can use the same too!

APFS Dirty volumes fail to process

If an APFS volume has not been cleanly unmounted, the resulting disk image cannot be processed by mac_apt. The following error will be seen:

MAIN-ERROR-Unknown error while trying to determine partition
MAIN-ERROR-Exception
Traceback (most recent call last):
  File "/Users/yogesh/Downloads/mac_apt_binaries_10.10/mac_apt.app/Contents/Resources/mac_apt.py", line 390, in <module>
  File "/Users/yogesh/Downloads/mac_apt_binaries_10.10/mac_apt.app/Contents/Resources/mac_apt.py", line 239, in FindOsxPartition
  File "/Users/yogesh/Downloads/mac_apt_binaries_10.10/mac_apt.app/Contents/Resources/mac_apt.py", line 177, in FindOsxPartitionInApfsContainer
  File "plugins/helpers/apfs_reader.pyc", line 669, in __init__
AttributeError: 'Node' object has no attribute 'root'
MAIN-WARNING-:( Could not find a partition having an OSX installation on it

parser IDEVICEINFO: value error

When parsing com.apple.iPod.plist with IDEVICEINFO, the script stumbles upon a Region Code which is not defined.

In my case this is ZD.

The script errors and quits.

This can be resolved by adding the line "Unknown Country = "ZD" in iDeviceInfo.py.
(After a fast Google, I didn't find any documentation about it to fill in the right country.)

Cellebrite Tar

Will this work with a Cellebrite tar? We opened the tar and there wasn't the iOS structure spoken about in the documentation.

No module named 'pyewf' when running mac_apt on High Sierra

So I followed the directions exactly three times and each time, I received the following error:

Traceback (most recent call last):
  File "mac_apt.py", line 26, in <module>
    import pyewf
ModuleNotFoundError: No module named 'pyewf'

Of course I followed the instructions regarding how to install - pyewf

Download libewf-20171104.tar.gz from https://github.com/libyal/libewf/releases Unzip and compile:

./configure --enable-python --with-pyprefix
make
sudo make install

So any thoughts? Thanks in advance!

ReadPlist should use biplist.readPlistFromString instead of biplist.readPlist

It turns out that biplist (which wraps plistlib) fails to parse plist files that have newlines and redundant white-spaces.

If you will replace biplist.readPlist with readPlistFromString, then we would be able to strip the newlines and white-spaces, and then we will not miss any plist file.

exported Notes lose formatting

Exported notes contain only the plain text content of the notes, losing any
rich text formatting. The formatting information seems to be stored separately
from the text that it applies to. It's not obviously in html or rtf or any other
common rich text format. It would be great if this formatting information could
be extracted and applied to the text, creating an html or markdown or similar
rich text document. The Exporter app is able to create a markdown version
of the text.

Some process names do not appear in the NETUSAGE plugin results.

In my test environment, netusage.sqlite on macOS 10.15.5, I found a case where an entry for a process is in the ZPROCESS table, but not in the ZLIVEUSAGE table.

In such a case, there is no data in ZWIFIIN, ZWIREDOUT, etc., but the ZFIRSTTIMESTAMP in the ZPROCESS table can be referenced. I consider ZFIRSTTIMESTAMP alone to be an important artifact.

Therefore, I would like to suggest the following change to the SQL query (replace "zprocess p" and "zliveusage lu" in the FROM clause).

SELECT pk.z_name as item_type ,p.zprocname as process_name, 
                p.zfirsttimestamp as first_seen_date, 
                p.ztimestamp as last_seen_date,
                lu.ztimestamp as usage_since, 
                lu.zwifiin, lu.zwifiout,lu.zwiredin,lu.zwiredout,lu.zwwanin,lu.zwwanout  
                FROM zprocess p LEFT JOIN zliveusage lu ON p.z_pk = lu.zhasprocess  
                LEFT JOIN z_primarykey pk ON p.z_ent = pk.z_ent  
                ORDER BY process_name

What do you think?

I'm also attaching a sample file.
netusage_sample.zip

The entry for "curl" is recorded in the ZPROCESS table, but not in ZLIVEUSAGE.
By the way, I don't know why, but the value of ZTIMESTAMP for curl is negative.

install script does not install all dependencies on mojave

FYI - Installed mac_apt v2 using installer script mac_aptInstall-macOS-v20.sh on mojave 10.14.6. Got a message mac_apt successfully downloaded and installed!. (same mac has working mac_aptV17)

However, while running python3 mac_apt.py u/ virtualenv ... it continue to complaint missing modules 1 at a time .... manually installed first 3 using pip ... stuck at lzfse as it does not have pip. Thanks

biplist
pytsk3
kaitaistruct
lzfse

macos install script failing on catalina 10.15.7

The install script throws error as follows

[+] Installing pyewf...
./mac_aptInstall-macOS-v21.sh: line 197: 15702 Abort trap: 6           wget https://github.com/libyal/libewf-legacy/releases/download/20140807/libewf-20140807.tar.gz >&/dev/null
[!] Download of pyewf failed.
[!] Please report this to the developer.

Tried to manually download above file wget https://github.com/libyal/libewf-legacy/releases/download/20140807/libewf-20140807.tar.gz. Noticed following error:

dyld: Library not loaded: /usr/local/opt/openssl/lib/libssl.1.0.0.dylib
  Referenced from: /usr/local/bin/wget
  Reason: image not found
Abort trap: 6

Any help ? Thanks