ciyfly / argo

Argo is an automated general crawler for automatically obtaining website URLs . Argo 是一个自动化扫描器爬虫 用于自动化获取网站的URL 基于go-rod实现了静态和动态结合的方式来实现

• 🔭 写点轮子

• 🌱

• 👀 微信公众号 欢迎关注 会分享安全研究 安全开发相关的内容

argo's Issues

问题

D:\tools\Argo-v1.2.9-windows>argo.exe -t https://192.168.1.1
[2023-05-15 18:58:30] [info] [argo start]
[2023-05-15 18:58:30] [info] target: https://192.168.1.1
[2023-05-15 18:58:33] [info] [GET] https://192.168.1.1/
[2023-05-15 18:58:36] [info] [tab count] 1
[2023-05-15 18:58:36] [error] No content crawled, you can contact the developer to recar target: 192.168.1.1
好多都是这样的问题

panic: sync: negative WaitGroup counter

./argo -f /tmp/domains.txt --format txt --proxy http://127.0.0.1:7777

似乎一出现超时就崩 [timeout tab ] => https://xxx.com/H7ocmnf12x

panic: sync: negative WaitGroup counter

goroutine 110 [running]:
sync.(*WaitGroup).Add(0xc0001286e0?, 0xc0000e9d68?)
        /opt/hostedtoolcache/go/1.19.11/x64/src/sync/waitgroup.go:83 +0xda
sync.(*WaitGroup).Done(...)
        /opt/hostedtoolcache/go/1.19.11/x64/src/sync/waitgroup.go:108
argo/pkg/engine.(*EngineInfo).closeTab(0xc0001a0000, 0xc0001286e0, 0x4, 0x3, 0x0?)
        /home/runner/work/Argo/Argo/pkg/engine/tab.go:57 +0x279
argo/pkg/engine.(*EngineInfo).NormalCloseTab(...)
        /home/runner/work/Argo/Argo/pkg/engine/tab.go:62
argo/pkg/engine.(*EngineInfo).NewTab.func1()
        /home/runner/work/Argo/Argo/pkg/engine/tab.go:88 +0x7e5
created by argo/pkg/engine.(*EngineInfo).NewTab
        /home/runner/work/Argo/Argo/pkg/engine/tab.go:81 +0x1aa

师傅，什么时候支持批量url列表

对于多个目标扫描时？

argo -f 1.txt - --format txt

1.txt里有
https://github.com/Ciyfly
https://github.com/Ciyfly/Argo

那么结果就会有2个文件夹，2个相同名字的txt
可以把结果都保存到一个文件内吗？不要多个文件

debian 10 vps 报错，内存充足

root@soapsbounty4H8Gdebian:~# ./argo -f alldomains_origin_urls.txt  --format txt
[2023-10-20 16:17:44] [info]  [argo start]
/2023-10-20 16:17:44] [info]  target: http://qq.travel
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x38 pc=0x90628f]

goroutine 1 [running]:
argo/pkg/req.getHttpClient({0xc0000be3c0?, 0xc0000e0040?}, {0xb32f55?, 0xc000062030?})
        /github/workspace/pkg/req/req.go:23 +0x4f
argo/pkg/req.CheckTarget({0xc0000be3c0?, 0x4?})
        /github/workspace/pkg/req/req.go:43 +0x47
main.RunMain(0xc0000d0160?)
        /github/workspace/cmd/argo.go:252 +0x28e
github.com/urfave/cli/v2.(*Command).Run(0xc0000d0160, 0xc000092080, {0xc00012c000, 0x5, 0x5})
        /go/pkg/mod/github.com/urfave/cli/[email protected]/command.go:273 +0xa42
github.com/urfave/cli/v2.(*App).RunContext(0xc0000a4000, {0xfd5d78?, 0xc00012e000}, {0xc00012c000, 0x5, 0x5})
        /go/pkg/mod/github.com/urfave/cli/[email protected]/app.go:332 +0x616
github.com/urfave/cli/v2.(*App).Run(...)
        /go/pkg/mod/github.com/urfave/cli/[email protected]/app.go:309
main.main()
        /github/workspace/cmd/argo.go:219 +0x1469
root@soapsbounty4H8Gdebian:~# ./argo -v
argo version v1.2.13
root@soapsbounty4H8Gdebian:~# 

收集问题

师傅们使用了有啥问题提奥

秀啊老铁

部分网站爬不到

PS C:\Users\LENOVO\Desktop\Argo\cmd> ./argo -t https://www.zuhaowan.com
[2023-05-04 10:18:02] [info]  [argo start]
[2023-05-04 10:18:02] [info]  target: https://www.zuhaowan.com
[2023-05-04 10:18:05] [info]  [GET] https://www.zuhaowan.com/
[2023-05-04 10:18:35] [warning]  [timeout tab ] => https://www.zuhaowan.com
[2023-05-04 10:18:36] [info]  [tab  count] 1
[2023-05-04 10:18:36] [error]  No content crawled, you can contact the developer to recar target: www.zuhaowan.com