微慑信息网

[工具]信息收集工具(js、url、domain)

(1)JSFinder(获取js中的URL)

https://github.com/Threezh1/JSFinder

用法:

  • 简单爬取
python JSFinder.py -u http://www.mi.com

这个命令会爬取 http://www.mi.com 这单个页面的所有的js链接,并在其中发现url和子域名

返回示例:

url:http://www.mi.com                                         
Find 50 URL:                                                  
http://api-order.test.mi.com                                  
http://api.order.mi.com                                       
http://userid.xiaomi.com/userId                               
http://order.mi.com/site/login?redirectUrl=                                                   
...已省略                            
                                                              
Find 26 Subdomain:                                            
api-order.test.mi.com                                         
api.order.mi.com                                              
userid.xiaomi.com                                             
order.mi.com                                                                                              
...已省略

  • 深度爬取
python JSFinder.py -u http://www.mi.com -d

深入一层页面爬取JS,时间会消耗的更长。

建议使用-ou 和 -os来指定保存URL和子域名的文件名。 例如:

python JSFinder.py -u http://www.mi.com -d -ou mi_url.txt -os mi_subdomain.txt
  • 批量指定URL/指定JS

指定URL:

python JSFinder.py -f text.txt

指定JS:

python JSFinder.py -f text.txt -j

可以用brupsuite爬取网站后提取出URL或者JS链接,保存到txt文件中,一行一个。

指定URL或JS就不需要加深度爬取,单个页面即可。

  • 其他

-c 指定cookie来爬取页面 例:

python JSFinder.py -u http://www.mi.com -c "session=xxx"

-ou 指定文件名保存URL链接 例:

python JSFinder.py -u http://www.mi.com -ou mi_url.txt

-os 指定文件名保存子域名 例:

python JSFinder.py -u http://www.mi.com -os mi_subdomain.txt
  • 注意

url 不用加引号

url 需要http:// 或 https://

指定JS文件爬取时,返回的URL为相对URL

指定URL文件爬取时,返回的相对URL都会以指定的第一个链接的域名作为其域名来转化为绝对URL。

 

 

 

(2)subjs(获取js)

https://github.com/lc/subjs/releases

用法:

Usage:

Examples:

$ cat urls.txt | subjs 
$ subjs -i urls.txt
$ cat hosts.txt | gau | subjs

To display the help for the tool use the -h flag:

$ subjs -h
Flag Description Example
-c Number of concurrent workers subjs -c 40
-i Input file containing URLS subjs -i urls.txt
-t Timeout (in seconds) for http client (default 15) subjs -t 20
-ua User-Agent to send in requests subjs -ua "Chrome..."
-version Show version number subjs -version"

(3)subfinder(子域名查找)

https://github.com/projectdiscovery/subfinder

 

用法:

Usage

subfinder -h

This will display help for the tool. Here are all the switches it supports.

Flag Description Example
-all Use all sources (slow) for enumeration subfinder -d uber.com -all
-cd Upload results to the Chaos API (api-key required) subfinder -d uber.com -cd
-config string Configuration file for API Keys, etc subfinder -config config.yaml
-d Domain to find subdomains for subfinder -d uber.com
-dL File containing list of domains to enumerate subfinder -dL hackerone-hosts.txt
-exclude-sources List of sources to exclude from enumeration subfinder -exclude-sources archiveis
-max-time Minutes to wait for enumeration results (default 10) subfinder -max-time 1
-nC Don’t Use colors in output subfinder -nC
-nW Remove Wildcard & Dead Subdomains from output subfinder -nW
-ls List all available sources subfinder -ls
-o File to write output to (optional) subfinder -o output.txt
-oD Directory to write enumeration results to (optional) subfinder -oD ~/outputs
-oI Write output in Host,IP format subfinder -oI
-oJ Write output in JSON lines Format subfinder -oJ
-r Comma-separated list of resolvers to use subfinder -r 1.1.1.1,1.0.0.1
-rL Text file containing list of resolvers to use subfinder -rL resolvers.txt
-recursive Enumeration recursive subdomains subfinder -d news.yahoo.com -recursive
-silent Show only subdomains in output subfinder -silent
-sources Comma separated list of sources to use subfinder -sources shodan,censys
-t Number of concurrent goroutines for resolving (default 10) subfinder -t 100
-timeout Seconds to wait before timing out (default 30) subfinder -timeout 30
-v Show Verbose output subfinder -v
-version Show current program version subfinder -version

 

(4)getJS

https://github.com/003random/getJS

用法:

Usage

Note: When you supply urls from different sources, e.g. with stdin and an input file, it will add all the urls together 🙂
Example: echo "https://github.com" | getJS --url https://example.com --input domains.txt

To get all options, do:

getJS -h
Flag Description Example
–url The url to get the javascript sources from getJS –url https://poc-server.com
–method The request method. e.g. POST or GET. Default: “GET” getJS –url https://poc-server.com –method POST
–timeout The request timeout. Default: 10 (secs) getJS –url https://poc-server.com –timeout 15
–insecure Skip SSL certificate verification. Use when the cert is expired or invalid getJS –url https://poc-server.com –insecure
–header Custom request header(s). -H “Authorization: Bearer token” getJS –url https://poc-server.com –insecure
–input Input file with urls getJS –input domains.txt
–output The file where to save the output to getJS –output output.txt
–verbose Display info of what is going on getJS –verbose
–complete Complete the urls. e.g. /js/index.js -> https://example.com/js/index.js getJS –complete
–resolve Resolve the output and filter out the non existing files (Can only be used in combination with –complete) getJS –complete –resolve
–nocolors Don’t color the output getJS –nocolors

 

(5)ksubdomain(域名爆破)

https://github.com/knownsec/ksubdomain/releases

用法:

releases下载二进制文件。

在linux下,还需要安装libpcap-dev,在Windows下需要安装WinPcap,mac下可以直接使用。

 _  __   _____       _         _                       _
| |/ /  / ____|     | |       | |                     (_)
| ' /  | (___  _   _| |__   __| | ___  _ __ ___   __ _ _ _ __
|  <    \___ \| | | | '_ \ / _| |/ _ \| '_   _ \ / _  | | '_ \
| . \   ____) | |_| | |_) | (_| | (_) | | | | | | (_| | | | | |
|_|\_\ |_____/ \__,_|_.__/ \__,_|\___/|_| |_| |_|\__,_|_|_| |_|

Usage of ./ksubdomain:
  -b string
        宽带的下行速度,可以5M,5K,5G (default "1M")
  -check-origin
        会从返回包检查DNS是否为设定的,防止其他包的干扰
  -d string
        爆破域名
  -dl string
        从文件中读取爆破域名
  -e int
        默认网络设备ID,默认-1,如果有多个网络设备会在命令行中选择 (default -1)
  -f string
        字典路径,-d下文件为子域名字典,-verify下文件为需要验证的域名
  -filter-wild
        自动分析并过滤泛解析,最终输出文件,需要与'-o'搭配
  -l int
        爆破域名层级,默认爆破一级域名 (default 1)
  -o string
        输出文件路径
  -s string
        resolvers文件路径,默认使用内置DNS
  -sf string
        三级域名爆破字典文件(默认内置)
  -silent
        使用后屏幕将仅输出域名
  -skip-wild
        跳过泛解析的域名
  -summary
        在扫描完毕后整理域名归属asn以及IP段
  -test
        测试本地最大发包数
  -ttl
        导出格式中包含TTL选项
  -verify
        验证模式

常用命令

使用内置字典爆破
ksubdomain -d seebug.org

使用字典爆破域名
ksubdomain -d seebug.org -f subdomains.dict

字典里都是域名,可使用验证模式
ksubdomain -f dns.txt -verify

爆破三级域名
ksubdomain -d seebug.org -l 2

通过管道爆破
echo "seebug.org"|ksubdomain

通过管道验证域名
echo "paper.seebug.org"|ksubdomain -verify

 

 

(6)naabu

https://github.com/projectdiscovery/naabu

用法:

▶ naabu -h

This will display help for the tool. Here are all the switches it supports.

Flag Description Example
c Worker threads for fqdn to ip resolution naabu -c 25
config Configuration file for naabu naabu -config naabu.conf
p Ports to scan (80,443, 100-200) naabu -p –
top-ports Top Ports to scan (default top 100 naabu -top-ports 1000
host host/domain/CIDR to scan ports for naabu -host 192.168.0.1/24
iL File containing list of hosts to enumerate ports naabu -iL hosts.txt
ports-file File containing ports to enumerate for on hosts naabu -ports-file ports.txt
exclude-cdn Skip full port scans for CDNs (only checks for 80,443) naabu -exclude-cdn
exclude-hosts Skip port scans for given hosts naabu -exclude-hosts 192.168.0.1/24
exclude-file Skip port scans for given hosts in file naabu -exclude-file exclude.txt
exclude-ports Skip port scans on hosts for given ports naabu -exclude-ports 22,80,443
nmap nmap scans to run on results (works with config file) naabu -nmap
nmap-cli nmap scans to run on results naabu -nmap-cli ‘nmap -sV’
o File to write output to (optional) naabu -o output.txt
json Write output in JSON lines Format naabu -json
rate Rate of port scan probes per requests naabu -rate 1000
interface Network Interface to use for port scan naabu -interface eth0
interface-list List available interfaces and public ip naabu -interface-list
no-color Don’t Use colors in output naabu -no-color
retries Number of retries for the port scan probe (default 3) naabu -retries 10
silent Print found ports only in output naabu -silent
source-ip Source IP naabu -source-ip 10.10.10.10
s Scan Type (s – SYN, c – CONNECT) naabu -s c
timeout Millisecond to wait before timing out (default 700) naabu -timeout 700
verify Validate the ports again with TCP verification naabu -verify
debug Enable debugging information naabu -debug
version Show version of naabu naabu -version
warm-up-time Time in seconds between scan phases (default 2) naabu -warm-up-time

(7)httpx

https://github.com/projectdiscovery/httpx

用法:

httpx -h

This will display help for the tool. Here are all the switches it supports.

Flag Description Example
H Custom Header input httpx -H ‘x-bug-bounty: hacker’
follow-redirects Follow URL redirects (default false) httpx -follow-redirects
follow-host-redirects Follow URL redirects only on same host(default false) httpx -follow-host-redirects
http-proxy URL of the proxy server httpx -http-proxy hxxp://proxy-host:80
l File containing HOST/URLs/CIDR to process httpx -l hosts.txt
no-color Disable colors in the output. httpx -no-color
o File to save output result (optional) httpx -o output.txt
json Prints all the probes in JSON format (default false) httpx -json
vhost Probes to detect vhost from list of subdomains httpx -vhost
threads Number of threads (default 50) httpx -threads 100
http2 HTTP2 probing httpx -http2
pipeline HTTP1.1 Pipeline probing httpx -pipeline
ports Ports ranges to probe (nmap syntax: eg 1,2-10,11) httpx -ports 80,443,100-200
title Prints title of page if available httpx -title
path Request path/file httpx -path /api
content-length Prints content length in the output httpx -content-length
ml Match content length in the output httpx -content-length -ml 125
fl Filter content length in the output httpx -content-length -fl 0,43
status-code Prints status code in the output httpx -status-code
mc Match status code in the output httpx -status-code -mc 200,302
fc Filter status code in the output httpx -status-code -fc 404,500
tls-probe Send HTTP probes on the extracted TLS domains httpx -tls-probe
content-type Prints content-type httpx -content-type
location Prints location header httpx -location
csp-probe Send HTTP probes on the extracted CSP domains httpx -csp-probe
web-server Prints running web sever if available httpx -web-server
sr Store responses to file (default false) httpx -sr
srd Directory to store response (optional) httpx -srd httpx-output
unsafe Send raw requests skipping golang normalization httpx -unsafe
request File containing raw request to process httpx -request
retries Number of retries httpx -retries
silent Prints only results in the output httpx -silent
stats Prints statistic every 5 seconds httpx -stats
timeout Timeout in seconds (default 5) httpx -timeout 10
verbose Verbose Mode httpx -verbose
version Prints current version of the httpx httpx -version
x Request Method (default ‘GET’) httpx -x HEAD
method Output requested method httpx -method
response-time Output the response time httpx -response-time
response-in-json Include response in stdout (only works with -json) httpx -response-in-json
websocket Prints if a websocket is exposed httpx -websocket
ip Prints the host IP httpx -ip
cname Prints the cname record if available httpx -cname
cdn Check if domain’s ip belongs to known CDN httpx -cdn
filter-string Filter results based on filtered string httpx -filter-string XXX
match-string Filter results based on matched string httpx -match-string XXX
filter-regex Filter results based on filtered regex httpx -filter-regex XXX
match-regex Filter results based on matched regex httpx -match-regex XXX

 

(8)linkfinder(在JavaScript文件中发现端点及其参数)

https://github.com/GerbenJavado/LinkFinder

用法:

Usage

Short Form Long Form Description
-i –input Input a: URL, file or folder. For folders a wildcard can be used (e.g. ‘/*.js’).
-o –output “cli” to print to STDOUT, otherwise where to save the HTML file Default: output.html
-r –regex RegEx for filtering purposes against found endpoints (e.g. ^/api/)
-d –domain Toggle to use when analyzing an entire domain. Enumerates over all found JS files.
-b –burp Toggle to use when inputting a Burp ‘Save selected’ file containing multiple JS files
-c –cookies Add cookies to the request
-h –help show the help message and exit
pip3 install -r requirements.txt

用法:

Short Form Long Form Description
-i –input Input a: URL, file or folder. For folders a wildcard can be used (e.g. ‘/*.js’).
-o –output “cli” to print to STDOUT, otherwise where to save the HTML file Default: output.html
-r –regex RegEx for filtering purposes against found endpoints (e.g. ^/api/)
-d –domain Toggle to use when analyzing an entire domain. Enumerates over all found JS files.
-b –burp Toggle to use when inputting a Burp ‘Save selected’ file containing multiple JS files
-c –cookies Add cookies to the request
-h –help show the help message and exit

 

 

 

 

 

(9)subdomain_shell

https://github.com/Mr-xn/subdomain_shell

用法:

run.sh domain.com

赞(0) 打赏
转载请附本站链接,未经允许不得转载,,谢谢:微慑信息网-VulSee.com » [工具]信息收集工具(js、url、domain)

评论 抢沙发

微慑信息网 专注工匠精神

微慑信息网-VulSee.com-关注前沿安全态势,聚合网络安全漏洞信息,分享安全文档案例

访问我们联系我们

觉得文章有用就打赏一下文章作者

非常感谢你的打赏,我们将继续提供更多优质内容,让我们一起创建更加美好的网络世界!

支付宝扫一扫打赏

微信扫一扫打赏

登录

找回密码

注册