* Package name : harpoon Version : no releases published https://github.com/Te-k/harpoon/issues/194 Upstream Contact: https://github.com/Te-k * URL : https://github.com/Te-k/harpoon * License : GPL-3 Programming Lang: Python Description : CLI tool for open source and threat intelligence harpoon doesn't have a long description upstream, but it's a recon tool with various backends. i personnally want to use it to answer questions like "who is this set of IPs doing nasty things to my server?" looking for answers like reverse DNS, geolocation, ASN, shodan information and so on. harpoon can do a lot more though, by tapping into various data sources like virustotal, haveibeenpwned, and many more. i do not believe there is an equivalent in Debian. maybe metasploit could be crafted to do *some* of this, but there's so many modules in metasploit nowadays that it's hard to tell. there *is* a shodan module, for example and there's a tool to do reverse DNS lookups, but no ASN lookup I could find. harpoon is designed with recon in mind specifically. there's even a separate toolset called harpoontools that give commandline tools to a basic set of tools: https://github.com/Te-k/harpoontools Just that basically answers most of my needs! Would be maintained under the python team, I suppose. Code is not the cleanest (and due for a refactor) but has unit tests and written by a friend of a friend.
Oh, and another thing I forgot: the harpoon wiki names a few alternatives to this. https://github.com/Te-k/harpoon/wiki#other-tools Of those, the following might be interesting: https://github.com/kpcyrd/sn0int https://github.com/aancw/Belati Out of band, someone also mentioned nmap as an alternative already packaged in Debian, but I figured nmap was a much more active tool than what I am looking for here. Finally, there's also this tool I found that does a *lot* of what I want: https://github.com/nitefood/asn ... but it's kind of a messy shell script, which is why I am hoping to package harpoon instead.
Another thing I forgot, the author wrote a good guide on their blog in: https://randhome.io/blog/2018/02/23/harpoon-an-osint-/-threat-intelligence-tool/
I started working on a Docker container for this, based on a Debian
container. To simplify the install (e.g. not require rebuilding lxml), I
pre-install a bunch of Debian packages for Python dependencies, but a
*lot* are missing.
The following are present:
python3-bs4 \
python3-dateutil \
python3-dnspython \
python3-geoip2 \
python3-ipy \
python3-lxml \
python3-maxminddb \
python3-phonenumbers \
python3-pip \
python3-psycopg2 \
python3-pyasn \
python3-pygments \
python3-tz \
python3-requests \
python3-selenium \
python3-setproctitle \
python3-simplejson \
python3-telethon \
python3-virustotal-api \
But the following are not:
"OTXv2",
"PyGitHub>=1.55",
"archiveis",
"censys==2.2.0",
"consolemd==0.5.1",
"fullcontact.py",
"greynoise>=1.2.0",
"passivetotal>=2.5.9",
"pybinaryedge==0.5",
"pycrtsh==0.3.11",
"pyhashlookup==1.2.1",
"pyhunter",
"pymisp==2.4.159",
"pypdns==1.3",
"pypermacc==0.1.1",
"pysafebrowsing==0.1.2",
"pysecuritytrails==0.1.3",
"pythreatgrid2==0.1.1",
"shodan",
"spyonweb==0.1",
"threatminer==1.0",
"tweepy>=3.8.0",
"zetalytics-api==1.0.1",
The following are available in Debian, but somehow pinned or declared in
a way that makes pip still install them
"beautifulsoup4==4.11.1",
"configparser",
"lxml==4.9.2",
"phonenumbers==8.12.4",
"simplejson==3.17.6"
"telethon==0.19.1.6",
So if we actually want everything in harpoon to actually work, there is
a *lot* of work to package dependencies here.
I've also looked for more alternatives, thinking "surely someone else fixed this in a more simple way". I looked mostly at https://kali.org/tools for now. Out of those, i have selected the following tools: https://www.maltego.com/ https://github.com/smicallef/spiderfoot https://github.com/SparrowOchon/dnsenum2 https://github.com/GoVanguard/legion https://github.com/owasp-amass/amass A.
More notes on my use case... harpoon serves this poorly, as I explain upstream here: https://github.com/Te-k/harpoon/issues/190#issuecomment-1798667942 Basically, harpoon has a good `intel` command to lookup the reputation of a single IP address on multiple plugins. But that's it: it works only a on a *single* IP address, not *multiple*. Also, it doesn't seem like it works very reliably on all backends. For example, even though the `vt` command works, it doesn't seem to hookup with the `intel` command. Effectively, what harpoon fundamentally is is a wrapper around many backend services. The most interesting I have found are: * asn and the asncount command in harpoontools: ASN to name mappings from https://ftp.ripe.net/ripe/asnames/asn.txt, ftp://archive.routeviews.org/datapath/YYYYMM/ribs/XXXX http://archive.routeviews.org/bgpdata/%d.%02d/RIBS (from pyasn package) * dns: simple reverse/forward DNS checks, not in intel either * ipinfo.io: provides ASN lookups, VPN/Tor/Proxy checks * pulsedive.com: tor, blocklists, cryptomining, threat reports * threatminer.org: unclear * tor: check tor exit lists, pulls https://check.torproject.org/torbulkexitlist on each call (!) * urlhaus.abuse.ch: more malware oriented, https://threatfox.abuse.ch more interesting but not implemented * virustotal (vt command): domain, IP reputation, history, API, free to use but rate limited unless a premium account is requested (note that there's a separate RFP for the vt-cli commandline, #1034826) Then there's a bunch more interesting resources that are not implemented yet but that are still interesting: * criminalip.io: abuse records, botnet, Tor, VPN, Proxy, Hosting, CDN, mobile, scanner checks, requires plan to do more https://github.com/Te-k/harpoon/issues/184 * crowdsec.net: federated collaborative IP reporting, free daily data source https://github.com/Te-k/harpoon/issues/199 * project honeypot: lists IPs that fell into a honeypot, https://github.com/Te-k/harpoon/issues/64 * proxycheck.io: simple API, Tor, Proxy, "type" (business, wireless, residential, etc), VPN check, https://github.com/Te-k/harpoon/issues/110 More services I found in my search that could be useful to tap for extra confirmations: * abuseipdb.com: abuse reports * dronebl.org: abuse reports of "infected machines", RBL * check.spamhaus.org: classic spammer database, RBL Alright, that's what I got so far! a.
Another, more minimalistic tool is simply a DNS resolver with support for looking up ASNs: https://github.com/natesales/q Specifically those options are interesting: -w Resolve ASN/ASName for A and AAAA records -R, --resolve-ips Resolve PTR records for IP addresses in A and AAAA records It's unclear how much batch support this supports and obviously it doesn't do much beyond that (e.g. no lookup on other databases or geoip...) a.