Releases: McSib/e621_downloader
1.7.2 Release
This is more of a QOL update for the codebase and the bugs that were persisting inside of it. I took the time to clean things up, get everything working a little bit smoother, as well as fixing critical high-severity bugs that were bricking portions of the program. You can view this as more of a hotfix rather than a full fledge release, and that will be evident in the release notes, but I feel this was a needed update, especially with how messy and hard it was to work on the codebase the more things got added to it. There is only a Windows build right now, but I'm looking to do Linux build for this version (and future versions) too, just need the time to sit down and set everything up for that. That might be a minute, but be patient. Enjoy this update, and have a smoother experience from here on.
I also want to note that I moved to a different TLS crate for making connections since the default one was causing "Checking if the connection is secure..." issues. This may or may not have fixed the issue, but so far it has in my testing. This may also help with the VPN issue, but that will have to be tested by those who use VPNs.
What's Changed
General
- Fixed critical bug where
is_deleted
could not be found when downloading pools [#67, #60, #62]. - Blacklists can now use
score:
. - Fixed bug where tags containing a space at the end would attempt to create invalid directories that resulted in crashes [#59, #58, #68].
Misc
- Rolled back workflow Ubuntu version to 18.4 as the latest causes issues with OpenSSL.
- Documentation has been expanded and given more detail [#69 ... #91].
- Tons of code cleanup that are listed in the pull request (#65) below.
- Turned
Config
andLogin
into singletons for easier use and access. - Imports are now organized across all files.
Grabber
now has all the code related to grabbing posts broken off into multiple functions, increasing readability greatly.- Converted the programming workflow of the entire project to Gitflow for a more structured software development cycle (also reducing bugs from leaking into the
main
branch).
Pull Requests
- Removed is_deleted from PoolEntry by @head-gardener in #61
- Crate toml updates. by @McSib in #63
- Update workflow for this branch and update crates. by @McSib in #64
- Codebase cleanup by @McSib in #65
New Contributors
- @head-gardener made their first contribution in #61
Full Changelog: 1.7.1...1.7.2
1.7.1 Release
Time has passed since my last update, hasn’t it? It has been a while, and life has taken months of my dev time on this project away. But I am back, and I have more things for the program to refine it even more. This will mostly contain bug fixes, cleaning up parts of the project that I have overlooked, and keeping the program nice and up to date. I am thinking about doing something new for the next update, but it will take time, so I will have to balance it carefully to ensure that I can get it finished. Beyond that, enjoy the update, and resume using the program, now with some fixes that got rid of some annoying bugs that has plagued a few users.
And yes, this version has a Linux build too. ;3
Release Notes:
- You can now set the naming convention of downloaded files (md5 or id). [#47]
- The search term will now shorten in the downloader progress bar if its length is greater than 25 characters.
- The directory named after the search term will now shorten on Windows if it crosses the
MAX_PATH
length a path can have (260 characters). [#53] - Fixed an incredibly rare bug where the
created_at
field in API responses would be null. - Removed an unused function that was overlooked.
- Updated the parser to report parsing errors.
- Fixed numerous security vulnerabilities tied to my dependencies (I recommend updating just for this reason).
Bug Fixes:
- Located possible cause for bug that crashes the downloader at the very start of parsing or downloading (read more about this below). [#48]
- Fixed bug where tag including a colon crashed the downloader. [#51] [#52]
- Fixed bug where the program suddenly crashed when grabbing a single explicit or questionable post while in safe mode. [#54]
Misc:
- Marked
regex
,smallvec
, andtokio
to their latest versions to fix security vulnerabilities. - Updated
indicatif
to0.16.2
indefinitely until0.17.0
releases with no performance impacts. - Updated
dialoguer
to0.10.1
. - Updated
log
to0.4.17
. - Updated
simplelog
to0.12.0
. - Updated
reqwest
to0.11.10
. - Updated
serde
to1.0.137
. - Updated
serde_json
to1.0.81
. - Enforced more intense field visibility rules for structs.
- Created an auto builder to determine if the build is passing once commits are pushed.
- Fixed many Clippy warnings.
- Added debug printing for grabbing tags so that they are traceable as well.
- Created a
rustfmt.yml
for rustfmt to format more to a unified style I work with (this may change in the future as I figure out new things to change with it).
Notice for users who use VPNs
I have had a recurring “bug” that has shown in my issues the last couple of months, and they tend to crop up right after a new release, so I am going to supply a new notice for those using VPNs to prevent this becoming issue spam. There are users who are experiencing crashes consistently when parsing, obtaining blacklist, or downloading. It is an issue that is consistent, and each person thus far have been using a VPN with no other noticeable cause linked. After a multitude of testing, I have concluded that users using VPNs will occasionally have either e621 directly or Cloudflare prompt for a captcha, or a test for whether you are a robot. Since my program does not support GUI, or no tangible way of handling that, it will crash immediately. I have looked for fixes to this issue and have yet to find anything. So, if you are using a VPN, be warned, this can happen. The current work around for this issue is switching locations in the VPN (if you have that feature) or disabling the VPN altogether (if you have that option). I understand it is annoying, and can be a pain, but this is all I can do until I come across a fix. Sorry for the inconvenience, and apologies if you are some of the users experiencing this issue.
What's Changed
Full Changelog: 1.7.0...1.7.1
1.7.0 Redesign Hotfix 1 Release
In light of an issue that popped up on my board yesterday (#45), I decided to take the time to make a hotfix release that adjusts the logging system to be much much more informative. There were questions I asked the person in that issue board that shouldn't have needed to be asked. With these adjustments to the logger, this will no longer be an issue.
With all of this provided below, I hope you have a good day. And if you experience an issue in the future, just share your log file and your issue and that should provide more than enough information for what might be happening.
NOTE:
I uploaded a Linux version of e621_downloader! This version is compiled on Arch Linux, so please let me know if it works for you or not, I will see about getting it compiled for others as well. If you are a Debian user or someone who uses a distro that is Debian based, let me know if this runs for you at all. I want to find a way to give up-to-date and ready to use Linux compilations of my downloader to users without them needing to compile the program. So far, this is one way I found. Let me know how it work, and I hope you enjoy the downloader!
Hotfix Notes:
- The logger now can print low-level information about your system, OS family, architecture, as well as executable information and DLL information.
- The logger will now be able to print out downloader specific information, such as the downloader name, version, working directory, etc.
- The logger will also print out login information, like username, API key, and whether or not you wish to download favorites. I would like to note, your API key is not visible in the
.log
file, the key is replaced with*
and is merely there to show that the information was provided from thelogin.json
. This is in the event there is a bug related to login issues. - The logger can now show image save paths and is allowed to share collection information, such as collection names (typically these are search terms), collection category (pools, sets, general searches, etc.), among others.
Misc:
- The release version setup has now been heavily altered to accommodate the crate version.
Pull Requests Accepted:
Full Changelog: 1.7.0...1.7.0-hotfix.1
1.7.0 Redesign Release
Well, the 1.7.0 release is out. There isn't much done with this release, at least on the overall feature changes. In this update, I wanted to focus on primarily changing the visuals. I got tired of the black-and-white console and decided that injecting some color would make a nice difference. I also wanted a better debug time with this program. So, to do that, I incorporated a logger. This logger will not only make things easier to read, but it will also export its print-outs into a nice .log
file for me to be able to read. If you have a bug in the future, all you have to do now is explain what's generally happening and then supply your log file. I can get a much deeper understanding of everything and what's happening from there. So with that, I hope you enjoy this new release!
Note: You need this new version if you wish to log in through the downloader as a recent change to e621 causes past versions to crash.
Release Notes:
- A new logging system for the program with the ability to save the logs to a log file.
- All-new color and design changes to the program, making it much more pleasing to the eye.
Bug Fixes:
- Fixed an issue with the pool name number being off when saving pool images. [#41]
- Fixed a bug where the order at which pools were saved was incorrect and often out of order. [#43]
- Fixed a crashing bug where the program would completely crash if the user logged in. [#44]
Misc:
- Did a massive code clean up that made everything generally faster and more concise to read through.
- Removed comments that no longer applied to the project code.
- Updated
base64-url
from1.4.8
to1.4.13
- Updated
reqwest
from0.11.0
to0.11.9
- Updated
serde
from1.0.118
to1.0.136
- Updated
serde_json
from1.0.61
to1.0.79
- Added
dialoguer
at version0.10.0
- Added
console
at version0.15.0
- Added
log
at version0.4.14
- Added
simplelog
at version0.11.2
Known Issues:
indicatif
will no longer be updated for some time as it currently has a performance issue that causes a 98% performance decrease in active release runtime. Currently, the version ofindicatif
used in the project is the closest up-to-date version without this performance impact.
Full Changelog: 1.6.1...1.7.0
1.6.1 In The Future Hotfix
So while the main release has been out, there has been some bugs popping up here and there, and I've been waiting for most of them to show, and I feel it's finally shown enough for me to warrant a new release. This is just a basic hotfix with bug fixes, nothing new or impressive, so enjoy!
Release Notes:
Bug fixes:
- Fixed a bug where the validator had to be either a tag or alias, giving no option to emergency exit encase of typo or invalid tag [#39]
- Fixed bug where the pool name for the image file conflicted with OS file and folder name restrictions, causing unexpected behavior and downright crashing on some occasions [#37] [#38]
- Fixed bug where the program would crash after not being able to figure out category a tag was [#40]
Misc:
- Backpedaled to a previous form of the safe mode selection as the current was starting a separate thread/output and letting the program run beyond the prompt itself, opening a window for bugs in the futures.
1.6.0 In the Future Release
The 1.6.0 In the Future release is here, and there was a lot I had altered, the list will contain everything that was altered, so be prepared for a very nerdy list. Have fun~
Also, before I fully get to the release notes, I must clarify that there is now a filter with e621. If you are not logged into E6, a filter (almost like a global blacklist) is applied with the entry young -rating:s
. This blacklist will nullify any posts that fall under it. If you wish to download any posts of this kind, you must log in and download it, otherwise, this filter will be there.
Release Notes:
- Downloader now works with the new API on e621 [#30]
- Blacklist system updated and reworked to provide much greater accuracy and performance [#34]
- Login system uses a much safer method to authenticate (the safer method was added with the new version of e621)
- The server now conforms to a new filtering system that nullifies the file URL to posts containing the tag
young
and not having the post rating set to safe [#31] - Users no longer need their ID in the
login.json
file to log into their accounts (this is a specific note from those who used theindev
release of 1.6.0) - The flagging system for the blacklist on the downloader now supports negated ratings, users, and IDs.
Bug fixes:
- Fixed massive memory leak in the blacklist system [#33]
- Fixed issue where the program would crash if the user's blacklist is empty [#35]
- Fixed issue where deleted posts and filtered posts (part of the new API) would crash the downloader while it was grabbing and downloading posts [#31, #32]
- Fixed bug where the user was required to log in, rather than logging in being an option
- Fixed bug where the blacklist would completely ignore blacklisted users
Misc:
- Added new depenancy
base64-url
at version1.2.0
for the new authentacation system - Updated
dialoguer
from0.5.0
to0.6.2
- Updated
indicatif
from0.13.0
to0.15.0
- Updated
reqwest
from0.10.1
to0.10.6
- Updated
serde
from1.0.104
to1.0.112
- Updated
serde_json
from1.0.44
to1.0.55
- Updated
dialoguer
from0.1.6
to0.1.8
1.6.0 Indev 2
This update is a massive rewrite and fix of the blacklist system. The previous version contained a huge memory leak I had not known about and came to the conclusion that the system needed to be updated which was stated here [#34]. This build should be completely usable.
This build is still in debug mode and will be until it is fully stable and released.
I hope you enjoy this, hopefully, more functional build.
1.6.0 Indev
This is a source mock-up of what is currently working in my software. This will only contain a debug build of the current version. No release builds are allowed right now as I need errors to print in large. This program will run slower than the more optimized release build, but in turn, it will allow you to see what is currently going on. There are also changes to the software that may take a moment to get used to, but this will all be leveled out when the build is stabilized. Till then, enjoy this debug build~!
1.5.6 Pools Release
Release Notes
- Pools are now downloaded in its entirety instead of just downloading one page.
- Pools' posts are now numbered when downloading, so reading them is much easier.
Tech-savvy Release Notes
- Better formating on
tags.txt
. - When a duplicate is skipped, it will now increment the
file_size
before continuing to the next post to download. - Removed the "Favorites" category when downloading the user's favorites.
- Documented code I forgot to document.
- Updated
reqwest
from 0.9.24 to 0.10.1. - Updated
serde
from 1.0.103 to 1.0.104 - The progress bar now fills the line it's on.
- Removed README.txt
1.5.5 Progress Release
Release Notes:
- Replaced the progress bar with a better one. This one can now show the total file size you need to download, and how far along you are.
- The prompt for when you want to enter safe mode is now handled through a better system.
Tech-savvy Release Notes:
- Fixed bug where the images became directories.
- Renamed function
get_length_posts
toget_file_size_from_posts
. - Made the
download_set
function much more readable. - Reverses array when grabbing posts instead of reversing them right before downloading them.
- Added new progress bar that displays the size of bytes that are going to be downloaded, instead of how many images.
- Small variable rename in
FlagWorker
- Fixed bug where the flags raised would pass the margin because the artist tags were being flagged twice.
Grabber
now manages theBlacklist
completely, not relying on theWebConnector
for the information anymore.- Blacklist is no longer created and parsed every time a new page of posts are grabbed.