saferwall / pe

A :zap: lightweight Go package to parse, analyze and extract metadata from Portable Executable (PE) binaries. Designed for malware analysis tasks and robust against PE malformations.

Portable Executable Parser

pe is a go package for parsing the portable executable file format. This package was designed with malware analysis in mind, and being resistent to PE malformations.

Table of content

• Portable Executable Parser
  • Table of content
  • Features
  • Installing
  • Using the library
    • PE Header
    • Rich Header
    • Iterating over sections
  • Roadmap
  • Fuzz Testing
  • References

Features

• Works with PE32/PE32+ file format.
• Supports Intel x86/AMD64/ARM7ARM7 Thumb/ARM8-64/IA64/CHPE architectures.
• MS DOS header.
• Rich Header (calculate checksum and hash).
• NT Header (file header + optional header).
• COFF symbol table and string table.
• Sections headers + entropy calculation.
• Data directories
  • Import Table + ImpHash calculation.
  • Export Table
  • Resource Table
  • Exceptions Table
  • Security Table + Authentihash calculation.
  • Relocations Table
  • Debug Table (CODEVIEW, POGO, VC FEATURE, REPRO, FPO, EXDLL CHARACTERISTICS debug types).
  • TLS Table
  • Load Config Directory (SEH, GFID, GIAT, Guard LongJumps, CHPE, Dynamic Value Reloc Table, Enclave Configuration, Volatile Metadata tables).
  • Bound Import Table
  • Delay Import Table
  • COM Table (CLR Metadata Header, Metadata Table Streams)
• Report several anomalies

Installing

Using this go package is easy. First, use go get to install the latest version of the library. This command will install the pedumper executable along with the library and its dependencies:

go get -u github.com/saferwall/pe

Next, include pe package in your application:

import "github.com/saferwall/pe"

Using the library

package main

import (
	peparser "github.com/saferwall/pe"
)

func main() {
    filename := "C:\\Binaries\\notepad.exe"
    pe, err := peparser.New(filename, &peparser.Options{})
	if err != nil {
		log.Fatalf("Error while opening file: %s, reason: %v", filename, err)
    }

    err = pe.Parse()
    if err != nil {
        log.Fatalf("Error while parsing file: %s, reason: %v", filename, err)
    }
}

Start by instantiating a pe object by called the New() method, which takes the file path to the file to be parsed and some optional options.

Afterwards, a call to the Parse() method will give you access to all the different part of the PE format, directly accessible to be used. Here is the definition of the struct:

type File struct {
	DOSHeader    ImageDOSHeader
	RichHeader   RichHeader
	NtHeader     ImageNtHeader
	COFF         COFF
	Sections     []Section
	Imports      []Import
	Export       Export
	Debugs       []DebugEntry
	Relocations  []Relocation
	Resources    ResourceDirectory
	TLS          TLSDirectory
	LoadConfig   LoadConfig
	Exceptions   []Exception
	Certificates Certificate
	DelayImports []DelayImport
	BoundImports []BoundImportDescriptorData
	GlobalPtr    uint32
	CLR          CLRData
	IAT          []IATEntry
	Header       []byte
	data         mmap.MMap
	closer       io.Closer
	Is64         bool
	Is32         bool
	Anomalies    []string
	size         uint32
	f            *os.File
	opts         *Options
}

PE Header

As mentionned before, all members of the struct are directly (no getters) accessible, additionally, the fields types has been preserved as the spec defines them, that means if you need to show the prettified version of an int type, you have to call the corresponding helper function.

fmt.Printf("Magic is: 0x%x\n", pe.DosHeader.Magic)
fmt.Printf("Signature is: 0x%x\n", pe.NtHeader.Signature)
fmt.Printf("Machine is: 0x%x, Meaning: %s\n", pe.NtHeader.FileHeader.Machine, pe.PrettyMachineType())

Output:

Magic is: 0x5a4d
Signature is: 0x4550
Machine is: 0x8664, Meaning: x64

Rich Header

Example:

richHeader, _ := json.Marshal(pe.RichHeader)
fmt.Print(prettyPrint(richHeader))

Output:

{
    "XorKey": 2796214951,
    "CompIDs": [
        {
            "MinorCV": 27412,
            "ProdID": 257,
            "Count": 4,
            "Unmasked": 16870164
        },
        {
            "MinorCV": 30729,
            "ProdID": 147,
            "Count": 193,
            "Unmasked": 9664521
        },
        {
            "MinorCV": 0,
            "ProdID": 1,
            "Count": 1325,
            "Unmasked": 65536
        },
        {
            "MinorCV": 27412,
            "ProdID": 260,
            "Count": 9,
            "Unmasked": 17066772
        },
        {
            "MinorCV": 27412,
            "ProdID": 259,
            "Count": 3,
            "Unmasked": 17001236
        },
        {
            "MinorCV": 27412,
            "ProdID": 256,
            "Count": 1,
            "Unmasked": 16804628
        },
        {
            "MinorCV": 27412,
            "ProdID": 269,
            "Count": 209,
            "Unmasked": 17656596
        },
        {
            "MinorCV": 27412,
            "ProdID": 255,
            "Count": 1,
            "Unmasked": 16739092
        },
        {
            "MinorCV": 27412,
            "ProdID": 258,
            "Count": 1,
            "Unmasked": 16935700
        }
    ],
    "DansOffset": 128,
    "Raw": "47vE9afaqqan2qqmp9qqprOxq6ej2qqmrqI5pmbaqqan2qumit+qprOxrqeu2qqms7Gpp6TaqqazsaqnptqqprOxp6d22qqms7FVpqbaqqazsainptqqplJpY2in2qqm"
}

Iterating over sections

for _, sec := range pe.Sections {
    fmt.Printf("Section Name : %s\n", sec.NameString())
    fmt.Printf("Section VirtualSize : %x\n", sec.Header.VirtualSize)
    fmt.Printf("Section Flags : %x, Meaning: %v\n\n",
        sec.Header.Characteristics, sec.PrettySectionFlags())
}

Output:

Section Name : .text
Section VirtualSize : 2ea58
Section Flags : 60500060, Meaning: [Align8Bytes Readable Align16Bytes Executable Contains Code Initialized Data Align1Bytes]

Section Name : .data
Section VirtualSize : 58
Section Flags : c0500040, Meaning: [Readable Initialized Data Writable Align1Bytes Align16Bytes Align8Bytes]

Section Name : .rdata
Section VirtualSize : 18d0
Section Flags : 40600040, Meaning: [Align2Bytes Align8Bytes Readable Initialized Data Align32Bytes]

...

Roadmap

• imports MS-styled names demangling
• PE: VB5 and VB6 typical structures: project info, DLLCall-imports, referenced modules, object table

Fuzz Testing

To validate the parser we use the go-fuzz and a corpus of known malformed and tricky PE files from corkami.

Projects Using This Library

Fibratus A modern tool for Windows kernel exploration and tracing with a focus on security.

References

• Peering Inside the PE: A Tour of the Win32 Portable Executable File Format by Matt Pietrek
• An In-Depth Look into the Win32 Portable Executable File Format - Part 1 by Matt Pietrek
• An In-Depth Look into the Win32 Portable Executable File Format - Part 2 by Matt Pietrek
• Portable Executable File Format
• PE Format MSDN spec
• DotNET format
• BlackHat 2011 - CONSTANT INSECURITY: (PECOFF) Portable Executable FIle Format

pe's Issues

Hello, may I ask if the driver's signature is inaccurate now?

I am sure it is a driver file that has been successfully signed, but the return of pe. IsSigned is indeed false

Here is my code

filename := "/mnt/c/demo/driver_x64.sys"
	pe, err := peparser.New(filename, &peparser.Options{
		Fast: true,
	})
	if err != nil {
		fmt.Printf("fail to open file! %v\n", err)
		return
	}
	defer pe.Close()

	err = pe.Parse()
	if err != nil {
		fmt.Printf("Failed to parse file! %v\n", err)
		return
	}

	if pe.Is32 {
		fmt.Println("x86")
	}

	if pe.Is64 {
		fmt.Println("x64")
	}

	if pe.IsDLL() {
		fmt.Println("DLL")
	}

	if pe.IsEXE() {
		fmt.Println("EXE")
	}

	if pe.IsDriver() {
		fmt.Println("DRIVER")
	}

	if pe.IsSigned {
		fmt.Println("Signed")
	} else {
		fmt.Println("Unsigned")
	}

Malware using fake NumberOfSymbols causes OOM during ParseCOFFSymbolTable

Hey,

In symbol.go:266 you preallocate a slice with the given value of pe.NtHeader.FileHeader.NumberOfSymbols, this can be abused by malware to force an OOM error by providing a fake and absurdly high value. I suggest adding a configurable/sensible limit to prevent this.

I forked and added a hardcoded value for my own use, but a new config option and/or a sensible limit may help others.

Thanks

I tried to identify the .Net PE file, but there was an error.

this is https://github.com/pandasec888/taowu-cobalt-strike/blob/master/script/SharpBypassUAC.exe

I just want to get his moduleName..
Used to label the file name.

package main

import (
    "fmt"
    peparser "github.com/saferwall/pe"
)

func main() {
    filename:=`C:\Users\a3ro\Desktop\6FAD3A96FE407982818CE27F73C78B8AC3B0902BD85F104DC85EEF092F4186DE.exe`
    pe, err := peparser.New(filename, nil)
    if err != nil {
        fmt.Printf("Error while opening file: %s, reason: %s", filename, err)
    }

    err = pe.Parse()
    if err != nil {
        fmt.Printf("Error while opening file: %s, reason: %s", filename, err)
    }
}

Inaccurate value of Authentihash

the value of Authentihash is different from virustotal. and Value overflow occurred which cause panic.

Returning multiple certificate chains in a PE file

Is there a way using this package to extract multiple certificate chains that are present in a PE file? I am currently looking at a PE file that has multiple certificate chains. However the Parse function seems to only result 1 of those chains with 2 certs, but seems to skip over the remaining cert chains. Is there a slight modification that can be made to get those? Or do you know of any other go packages that currently do that?

Finally, great job with this package! It was really easy to use compared to some other libraries that I found :)

func: adjustSectionAlignment error (helper.go: 423)

should:

func (pe *File) adjustSectionAlignment(va uint32) uint32 {
	var fileAlignment, sectionAlignment uint32

	switch pe.Is64 {
	case true:
		fileAlignment = pe.NtHeader.OptionalHeader.(ImageOptionalHeader64).FileAlignment
		sectionAlignment = pe.NtHeader.OptionalHeader.(ImageOptionalHeader64).SectionAlignment // here
	case false:
		fileAlignment = pe.NtHeader.OptionalHeader.(ImageOptionalHeader32).FileAlignment // here
		sectionAlignment = pe.NtHeader.OptionalHeader.(ImageOptionalHeader32).SectionAlignment
	}

	if fileAlignment < FileAlignmentHardcodedValue &&
		fileAlignment != sectionAlignment {
		pe.Anomalies = append(pe.Anomalies, ErrInvalidSectionAlignment)
	}

        if section_alignment < 0x1000:  # page size
            section_alignment = file_alignment

	// 0x200 is the minimum valid FileAlignment according to the documentation
	// although ntoskrnl.exe has an alignment of 0x80 in some Windows versions
	if sectionAlignment != 0 && va%sectionAlignment != 0 {
		return sectionAlignment * (va / sectionAlignment)
	}
	return va
}

Integer divide by zero when checking against section alignment anomaly

While running a fuzzing session using the corkami corpus (corpus/) a certain file triggers a divide by zero panic when computing the modulo against section alignment of the file.

Resulting crash :

panic: runtime error: integer divide by zero

goroutine 1 [running]:
github.com/saferwall/pe.(*File).ParseNTHeader.func6(...)
	/home/echo/projects/saferwall/pe/ntheader.go:444
github.com/saferwall/pe.(*File).ParseNTHeader(0xc0000ac240, 0x0, 0x0)
	/home/echo/projects/saferwall/pe/ntheader.go:444 +0xf3e
github.com/saferwall/pe.(*File).Parse(0xc0000ac240, 0xc0000ac240, 0x604e02c3)
	/home/echo/projects/saferwall/pe/file.go:129 +0x127
github.com/saferwall/pe.Fuzz(0x7f0e504b5000, 0x138, 0x138, 0x4)
	/home/echo/projects/saferwall/pe/fuzz.go:10 +0x146
go-fuzz-dep.Main(0xc00004ff70, 0x1, 0x1)
	go-fuzz-dep/main.go:36 +0x1b8
main.main()
	github.com/saferwall/pe/go.fuzz.main/main.go:15 +0x52
exit status 2

Code responsible :

// The msdn states that SizeOfImage must be a multiple of the section
	// alignment. This is not true though. Adding it as anomaly.
	if (pe.Is32 && oh32.SizeOfImage%oh32.SectionAlignment != 0) ||
		(pe.Is64 && oh64.SizeOfImage%oh64.SectionAlignment != 0) {
		pe.Anomalies = append(pe.Anomalies, AnoInvalidSizeOfImage)
	}

This issue has been fixed and is here as reference.

Allow access to the raw certificates content

saferwall/pe provides access to the parsed certificates. However, neither saferwall/pe nor the used pkcs7 module allow access the to the raw certificates content.

I would like to read the raw certificates content using saferwall/pe, to persist and analyze it externally.

Avoid certutil for downloading certificates

Hi,

I'm so happy I've found this package/library! Great work. I'm planning to sunset the PE introspection functionality I've initially built into Fibratus in favour of your package. I was glancing at the code and noticed you rely on certutil to fetch the certificates you later use for validation. Since certutil is frequently abused by threats actors for nefarious purposes, I'm wondering if there is a way to download the certificates by interacting with some specific Windows API?

Remove logging to standard output

saferwall/pe makes use of the log package in exception.go which prevents me from using it in a CLI application:

2022/03/31 03:14:39 Wrong unwind opcode 15
2022/03/31 03:14:39 Wrong unwind opcode 15
2022/03/31 03:14:39 Wrong unwind opcode 14
2022/03/31 03:14:39 Wrong unwind opcode 13
2022/03/31 03:14:39 Wrong unwind opcode 12
…

saferwall/pe should not use logging on its own, as this often results in unexpected behavior.

The question is: shall we just drop that line and the log depedency as it is just a warning or do we need some logger-Interface/callback methods to be passed in the constructors?

memory leak

https://github.com/saferwall/pe/blob/main/file.go：

should：

// Close closes the File.
func (pe *File) Close() error {
	if pe.data != nil {
		_ = pe.data.Unmap()
	}

	if pe.f != nil {
		return pe.f.Close()
	}
	return nil
}

Make get_data / get_rva_from_offset public

feat: parse dotnet metadata tables

`Authentihash` Function doesn't generate valid Authenticode Hashes

I was interested in re-creating sbsign tool in golang, the only obstacle is that Authenticode hashes seem to be a headache. I found this library which looks very promising.

Am pretty sure there's an issue with the Authentihash function. I created a digest for a specific file using this library and then created another hash for the same file using: https://git.kernel.org/pub/scm/linux/kernel/git/jejb/sbsigntools.git/tree/src/image.c#n510

I got different values. I'm pretty confident on the result of sbsign, I signed my kernel with that and UEFI secure boot verifies it. I'm sorry I can't provide much more details or logs. I would like to be more helpful than just pointing things out. I thought it was still worth opening this.

Cheers.

UnitTests: reach 90% of code coverage

#74

loglevel setting

I want to disable logging like this: "Failed to parse data directory Relocation, reason: invalid relocation information. Base Relocation VirtualAddress is outside of PE Image" but i didn't found out how to do it?

I want to pass in byte[] file for memory scanning..

I want to pass in byte[] file for memory scanning.. You know
test

func NewBytes(buffer []byte, opts *Options) (*File, error) {

	tmpfile, err := ioutil.TempFile("", "example")
	_, _ = tmpfile.Write(buffer)

	// Memory map the file insead of using read/write.
	data, err := mmap.Map(tmpfile, mmap.RDONLY, 0)

	file := File{}
	if opts != nil {
		file.opts = opts
	} else {
		file.opts = &Options{}
	}
	file.data = data
	file.size = uint32(len(file.data))
	file.f = tmpfile
	return &file, nil
}

run in goroutine

When the code is executed to print end, the file is still occupied and cannot be deleted

func main() {
	var wg sync.WaitGroup
	wg.Add(1)

	go func() {
		filename := "C:\\test.exe"
		pe, err := peparser.New(filename, &peparser.Options{Fast: true})
		if err == nil {
			pe.Close()
		}
		wg.Done()
	}()

	wg.Wait()
	fmt.Println("end")
}

How to get the certificate signature time of the pe file

How to get the certificate signature time of the pe file???

feat: Maintain the state of which data directory has been parsed

This will help on cases where someone has chosen explicitly a fast-load option, then later tried to parse a directory that depended on another directory.

after running the pods internal error 500

Hi,

after running the pods i get an interanal error 500. cannot access ui

Incorrect check for skipCertVerification in security.go

The check on line 302 in security.go should be "if !skipCertVerification" so that VerifyWithChain will be called.

feat: support parsing version-information resource

https://learn.microsoft.com/en-us/windows/win32/menurc/versioninfo-resource

make pe.getOffsetFromRva public function

hi ,
would it be possible to rename pe.getOffsetFromRva to pe.GetOffsetFromRva so that one can get the offset in file of a specific rva ?
Thanks :)

Feat: extend cmd to display all parsed PE data

• Load Config

Printing to stdout instead of returning an error or logging

It looks like there is some code that can output to stdout when using this library:

pe/dotnet.go

Line 730 in 712c831

fmt.Printf("unhandled metadata table %d %s offset 0x%x cols %d\n", tableIndex, MetadataTableIndexToString(tableIndex), offset, table.CountCols)

In anchore/syft you can see this with:

syft -o json --platform windows/amd64 registry:mcr.microsoft.com/windows/servercore:ltsc2019 > /tmp/file.txt

# cat /tmp/file.txt | more
unhandled metadata table 38 File offset 0x944426 cols 5
unhandled metadata table 38 File offset 0xafdd82 cols 5
unhandled metadata table 38 File offset 0x1e720 cols 1
unhandled metadata table 38 File offset 0x3015aa cols 5
unhandled metadata table 38 File offset 0x1e718 cols 1
unhandled metadata table 38 File offset 0x30171e cols 5
unhandled metadata table 38 File offset 0x45a cols 1
unhandled metadata table 38 File offset 0x45a cols 1
unhandled metadata table 38 File offset 0x45a cols 1
unhandled metadata table 38 File offset 0x45a cols 1
unhandled metadata table 38 File offset 0x45a cols 1
unhandled metadata table 38 File offset 0x45a cols 1
unhandled metadata table 38 File offset 0x43c cols 1
unhandled metadata table 38 File offset 0x44470 cols 1
unhandled metadata table 38 File offset 0x92d34e cols 5
unhandled metadata table 38 File offset 0x583f4 cols 1
unhandled metadata table 38 File offset 0xadac8a cols 5
{
 "artifacts": [
  {
   "id": "845df080a5c1592e",
   "name": " ",
   "version": "10.0.0.0",
   "type": "dotnet",
   "foundBy": "dotnet-portable-executable-cataloger",
...

Ideally there shouldn't be printing to stdout in library calls, instead returning an error or writing directly to a logger object would be preferred. The reason why is because the stdout from syft is expected to be valid JSON, which this prevents (where a logger would write this to stderr).