Jump to content

Recommended Posts

Spent some time with the DNS seeders and there is very little info so I was playing with cloudflare and wondered if I could automate the DNS seeds somehow. I assume this works on any coins that has a peer list with minor tweaks. The basic idea is getpeerinfo from the daemon into a json file and then send line by line of that into an A record on your seeder domain name. I am hopeful some others will see this and have a better idea how to automate this by making it easier to setup and run on generic coin.

https://github.com/buzzkillb/duct-tape-dns-seeder

Make a cloudflare account and point your domain denarius.pro at the cloudflare nameservers from your domain host control panel. Now we can edit records on cloudflare and the changes are almost immediate.

Cloudflare API Key is here, top right Icon -> My Profile -> View Global API Key

image.thumb.png.9ef7975643a1190504a48daf3080dca4.png

#Install Python Cloudflare

sudo apt install python-pip
git clone https://github.com/cloudflare/python-cloudflare
cd python-cloudflare
./setup.py build
sudo ./setup.py install

#Create a config file for your cloudflare API, change email and token (API KEY)

mkdir ~/.cloudflare
nano ~/.cloudflare/cloudflare.cfg
[CloudFlare]
email = <user@example.com>
token = <API KEY>
certtoken = v1.0-...
extras =

#test this works. change the ipv4 and denarius.pro to your stuff. dnsseed.denarius.pro is what my example will show.

cli4 --post name="dnsseed" type="A" content="73.218.220.108" /zones/:denarius.pro/dns_records

now we want to store a couple text files somewhere. you choose this for now I will use /root/

#create seed.sh and edit denarius.pro to your domain name. still using dnsseed.denarius.pro for this example.

#!/bin/sh
grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' "${1:?}" |
while read IP
do
    echo "$IP"
    cli4 --post name="dnsseed" type="A" content="$IP" /zones/:denarius.pro/dns_records
done

make this file executable

chmod +x seed.sh

now how to grab and put the ipv4's into the domain A records. denariusd daemon send peerinfo into a json file. then jq parses the json for the addr array and then we remove some junk and put that into peers.txt. From there the bash file uses regex to make the ipv4's pretty.

denariusd getpeerinfo > peer.json
jq '.[] | .addr' -r peer.json | sed 's/[][]//g' > peers.txt
./seed.sh peers.txt

This can be updated as much as the cloudflare API limits gives. But how to remove A records and try to keep this list fresh?

Lets make a delete file from python cloudflare examples. https://github.com/cloudflare/python-cloudflare/blob/master/examples/example_delete_zone_entry.py

#create delete.py and chmod+x this, and then put this inside.

#!/usr/bin/env python
"""Cloudflare API code - example"""

from __future__ import print_function

import os
import sys
import re
import json
import requests

sys.path.insert(0, os.path.abspath('..'))
import CloudFlare

def main():
    """Cloudflare API code - example"""

    try:
        zone_name = sys.argv[1]
        dns_name = sys.argv[2]
    except IndexError:
        exit('usage: example_delete_zone_entry.py zone dns_record')

    cf = CloudFlare.CloudFlare()

    # grab the zone identifier
    try:
        params = {'name':zone_name}
        zones = cf.zones.get(params=params)
    except CloudFlare.exceptions.CloudFlareAPIError as e:
        exit('/zones %d %s - api call failed' % (e, e))
    except Exception as e:
        exit('/zones.get - %s - api call failed' % (e))

    if len(zones) == 0:
        exit('/zones.get - %s - zone not found' % (zone_name))

    if len(zones) != 1:
        exit('/zones.get - %s - api call returned %d items' % (zone_name, len(zones)))

    zone = zones[0]

    zone_id = zone['id']
    zone_name = zone['name']

    print('ZONE:', zone_id, zone_name)

    try:
        params = {'name':dns_name + '.' + zone_name}
        dns_records = cf.zones.dns_records.get(zone_id, params=params)
    except CloudFlare.exceptions.CloudFlareAPIError as e:
        exit('/zones/dns_records %s - %d %s - api call failed' % (dns_name, e, e))

    found = False
    for dns_record in dns_records:
        dns_record_id = dns_record['id']
        dns_record_name = dns_record['name']
        dns_record_type = dns_record['type']
        dns_record_value = dns_record['content']
        print('DNS RECORD:', dns_record_id, dns_record_name, dns_record_type, dns_record_value)

        try:
            dns_record = cf.zones.dns_records.delete(zone_id, dns_record_id)
            print('DELETED')
        except CloudFlare.exceptions.CloudFlareAPIError as e:
            exit('/zones.dns_records.delete %s - %d %s - api call failed' % (dns_name, e, e))
        found = True

    if not found:
        print('RECORD NOT FOUND')

    exit(0)

if __name__ == '__main__':
    main()

to run the deleter, and it appears this only deletes 10-15 records at a time, so you might need to run this 5 times before sending a fresh list. This is only deleting records from dnsseed.denarius.pro. Nothing else on denarius.pro. Magical.

./delete.py denarius.pro dnsseed

Right now I am trying to think how frequent to send new ip's and delete the list and start over. Once I get that down I will post a sample cronjob to use. Otherwise this should work with basically any bitcoin fork daemon, maybe minor tweaks.

I also need a better regex to parse ipv6 so we can also make some on the fly AAAA records.

Use the github as that shows the crontab for adding and deleting the A records


Join Denarius Discord - https://discord.gg/JQEmXwb

Share this post


Link to post
Share on other sites

I'm sure its a long way from being "perfect", but take a look at the generic-seeder project I started earlier this year and see if it doesn't already do more-or-less what you are looking for: https://github.com/team-exor/generic-seeder

I struggled way more than should have been necessary when I first started looking at how to adapt the bitcoin seeder to work with another coin. All the data that needs to change from coin to coin is buried in the source code in the most unobvious of places. So the first thing I did to remedy this was to add the ability to enter the kind of data that changes from coin to coin into a config file. That made it easy to quickly set up a new seeder for any coin in a matter of minutes (assuming you already know the current protocol version, port #, magic bytes, etc. for your coin).

The next thing I struggled with was setting up the actual dns record to make proper use of the data that the crawler was collecting and dumping. I see you already have what looks like a great set of instructions to do the DNS setup here: https://denariustalk.org/index.php?/topic/314-dns-seeder-setup/. However, I did find that some savvy coder(s) figured out a much easier and fool-proof way to accomplish the same thing by utilizing a python script and retasking a cloudflare account into performing the DNS portion of the seeders work (as sort of mentioned above this post). In an ideal world, the code for this cloudflare setup should not be in a separate python script but should instead be part of the c++ seeder code to limit dependencies on other programming languages. I hope to either find the time to build this out more properly in the future or maybe some kind soul will do that for me given that it is open source after all. I did take the liberty of modifying the original cloudflare script(s) I found in the wild by fixing a few bugs and integrating the configuration for the python script into the same config file that is consumed by the generic-seeder app - so there's no need for separate config files all over the place.

At this point, everything was working quite beautifully I thought, but still one thing bothered me about the original dns seeder code. I couldn't seem to understand why the block height that was being used to determine which nodes were considered "good" or "bad" was a static value. Now, I've been around a few blockchains over the years and one thing I've found that many chains suffer from are peers that are stuck on a block. To be honest, I never bothered to ask the original author of the bitcoin seeder why the block height check was based on a value that doesn't change. Perhaps there is a good reason for the way this was originally developed, and it could be that my next "fix" won't be appreciated or desired. But I decided that I wanted my seeder to effectively have the ability to weed out these pesky stuck peers and so the generic-seeder can also contact a trusted block explorer to achieve this goal. If a block explorer api url is specified in the config, the explorer api  is contacted every 60 seconds (this should likely be a config variable for next update) and the updated block height value from the explorer is then used to validate against other nodes in the next series of tests. The block explorer integration is optional and the seeder still has the ability to hardcode the block height (using the config file) if you so choose.

Feel free to clone or fork and build your own implementations on top of this. Again, it may certainly not be perfect, but I think it's a huge step in the right direction if I do say so myself. I hope you agree 😃

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...