I spent a lot of time on my homelab. On provisioning, and on the network. Have yet to finish either, though. More details on that when I achieve a semi worthy MVP. I am going to try to get better at less “complete” posts. More in the form of thoughts/ideas than a complete A-Z thing. I’ve got a lot of WIP stuff, but quite a bit more trial and error remains before completion.
The reasoning for this post, and others like it, is to have something to look back on later. Did I learn as much as I wanted to? Did I take another route? If someone finds it interesting, awesome. The primary audience is however myself in the future.
There is limited metrics here, only whatever CloudFlare Pages/DNS provide for now. Guessing it’s me, crawlers, and some friends.
And just for the record: I am ignoring the difference between programming and scripting here.
I have heard about "todo apps" being a common application to build to learn a language. You have to take user input, store it, and allow the user to edit/remove it.
My approach is not too different, just something I find more interesting. I want it to be reasonably scalable in terms of functionality and presentation.
In checking for subdomains, you have a list of strings to suffix to the apex domain and see if you get HTTP 200 or maybe HTTP 301/302. Then you keep track of the results, and present them in some way. You can also have some pause/resume option, improve speed by checking several at the time, as long as you are nice to the server considering load. It is not a DoS tool.
This can use a database connection, and model-view-controller approach.
I started with a [0]Bash script, CLI tool. This is currently rather basic, but "good enough."
Next up was [1]Python.
At first I planned to clone the structure from Bash, but quickly pivoted to try more of a framework than script. My OOP experience is limited, so it makes sense to focus a lot on that. Would also like to get skilled at PostgreSQL, but when starting out, SQLite is used. With some luck, the SQL syntax itself is similar or identical, and only the connection to the DB that must be modified later.
The core functionality will be running requests towards foo.bar.tld and keeping track of the results. With the list of subdomains being finite, some kind of status should be possible, like 50/350 checked, 3 found: a, b, c with continuous refreshing and maybe "3 requests / second" if I can math that together. The idea is to have it working, cause that’s fun. But also potentially overkill in the "nice to have" department, simply to include as many things I want to get experience with as I can.
As long as new stuff is done iteratively, with the entire product being functional (even if limited) along the way, it should remain fun. The longer it takes before it is even remotely useful/functional, the harder it is to keep motivated. Frequent victories with it working after each feature is added, mitigates this.
This has also been a factor in non-programming based projects. Cannot finish part A before part B is done, and part B depends heavily on C. Building a complete city before using any buildings within it as they are finished, is exhausting. Seeing the benefit (yes, I avoided using the fucking word "value") of what you are building during the process of making it is wonderful.
Losing interest in it after a while may still absolutely happen, but archiving it feels better when it’s working (even with an unfinished part in development), than a massive thing that is halfway done and completely useless.
[0] https://gitlab.com/m15n/subdomain_scan/-/tree/main/bash?ref_type=heads
[1] https://gitlab.com/m15n/subdomain_scan/-/tree/main/python?ref_type=heads