View source for Project:Maintenance/041325

== Maintenance Log ==
;[[Main Page|Return to the main page]] or
;[[Project:Maintenance|Return to the Maintenance Index]]

;4-7 notes
:Primarily spending time investigating issues surrounding PHP and its fpm-pool configuration. Settled upon a workable state.
:Reviewing other aspects of PHP fine tuning. Tidied up MediaWiki's LocalSettings - most gains outside of a redis / memcached approach have been explored.

;4-8 notes
:Recent string of updates & upgrades combined with some of the PHP tweaking has resulted in better overall performance.
:Navigating OPcache and monitoring changes to the pool config. 
:Looking over the rewrites & 404 handling. Need to revisit some aspects of the nginx config...
:'''I will try''' to implement the use of Varnish. It, some further PHP tuning and the aforementioned memory-oriented caching are essentially what remains of this topic.
:*"Ok, so what's going on with the file archive?"
::In brief, it dawned upon me I need a file ''conversion'' pipeline (not necessarily as a part of the site). The files I intend to serve, the formats I find them in and the process of reorganizing the files to serve the correct one at the correct place is something I can't handle haphazardly. I've tested a few services and built my own tool. That said, I like https://cloudinary.com
:Yes...I am finally looking into getting the e-mail server working '''correctly.'''
:Because the migration is much more than repopulating a URL with some new content I'll also be revisiting extensions and the status of the Citizen skin in the coming week. Adding on Varnish, making sure things like cache control, gzip's "config"/calls, text compression, wasted CSS and un-optimized resource loads are combed over. Lots of little this and that to do. This is a case where the first 80% is everything and the final 20% is just going through the motions.
:Needless to say I'm avoiding committing to any framework or backend service as a foundation for the file archive. As outlined before the functionality of the site overrides the form it takes, for now. Is it ugly? You bet your sweet cheeks it is. Does it work? Sure does, it even throws errors (when it's supposed to!). Naturally I'm leaning into PHP and towards Laravel. My taste for javascript-centric frameworks has already run thin.
*OPcache config in play. Monitoring.

;4-9 notes
:Curious behavior from the server. New settings have resulted in both some expected and other unexplained behavior. Reviewing logs. Possibility of strangeness on DigitalOcean's end of things not specifically site-related. (Determined to be recursive response from SMW, should be fixed via below actions).
:*Reported an issue with SMW 5.0.0 - incorrect maintenance pathing found in tangentially related slow responses.
::*Fixed the issue. Not specifically SMW's fault. Has to do with how composer installs. More verbosity in dependency updates corrects the failed maintenance call. Much weirder than it sounds.
:*Revamped OPcache and broader PHP settings. 99.4% hit rate last checked with solid headroom.
:*Varnish, memcached & redis have now hit the backlog. If PHP performance due to the recently committed changes continues its positive trends then I'll be refocusing on nginx & MediaWiki-specific optimization before re-approaching those issues. The current server configuration allows a healthy amount of memory to work with so there's room for improvement here.
:Later tonight I'll be marking a checkpoint for configuration. Thanks to a slow update schedule in late March & April and the recent performance tuning bearing fruit I can pivot back towards the file archive & interactive map. Original April schedule is entirely in-tact.

;4-11 notes
:Resuming work on file archive & interactive map. Recent spell of uptime has been overall positive. Reading up on Redis and will experiment with in on the sandbox server.
:Dialed in OPcache & APCu. Happy with the hit rate thus far. Pretty much rounds out the main body of PHP tuning, for now. Slow logs and such are being monitored for the more concerning stuff.
:Applied some unbelievably important adjustments to the nginx configuration. Asset handling & caching are solidly in place.

;4-12 notes
:Final review before migration. Visible activity has been quiet due to continued backend maintenance and work on the sandbox server.
::Nginx - 95% complete. Final touches on the site-wide config. 
::PHP - General configuration and performance above satisfactory.
::OPcache & APCu - Hitting and caching appropriately.
::Peculiar memory activity last night.

;4-13 notes
:Reorganizing small elements like the header, footer, testing search & tag filtering, icon display and grid responsiveness. 
:I'll be '''consolidating''' to a singular droplet for the three major sites (wiki, map and file archive).
:Once the transfer has been completed the old droplet will be snapped & closed. There may be unexpected & unannounced periods of downtime throughout today due to the staged rollout. Many backups & snapshots exist and will be created (including database dumps) to ensure this process unfolds smoothly. Early April linux updates have resulted in a stable environment and makes for a good time to commit to this effort.
# '''The map has been moved to:''' https://map.kenshi.wiki/
# '''The old wiki site will no longer be available (as it was).''' Currently converting https://kenshidb.wiki
# '''The migration is now complete with https://kenshidb.wiki now serving the file storage & archive.

;I'll now monitor site performance over the week and begin the rollout of updates to the new file storage site and interactive map.
;Final touches on the foundational wiki configuration will take place over the next two weeks in preparation for content oriented design.
;The migration and conversion process has finished. The old droplet has been closed and (as per testing) all new domains are functional. Updates will continue though the bulk of initial site maintenance  and prep-work has reached a major milestone. What remains is now further performance improvements and advanced API integration.