Fullscreen
TikiWiki Remote Instance Manager (Cached)
Refresh Print

TRIM

TRIM is an acronym for the TikiWiki Remote Instance Manager

Documentation is at doc:TRIM


Table of contents



Current state

State as of April 8th, 2009:
  • It is not yet for production servers. Use at your own risk. However, it was demonstrated to work on different hosts, so somewhat usable.
  • It is somewhat usable in "ideal" conditions (ideal PHP configuration, certain tools must be installed on the server), but not yet robust to deal with various server configurations.
  • At least 3 TRIMs are running to manage a total of over 30 TikiWiki sites
  • It is not yet fully documented. It is a wizard-like app, so if you are familiar with shell scripts and installing/upgrading TikiWiki, you should be OK. However, it is not robust to user errors. For example, if you mistype your database username, the script could just die instead of giving a second chance.
  • In active development. Make sure to update regularly and report issues / improve the documentation.


What soon/maybe

  • For now, it requires SSH access but it is designed so it could eventually work for FTP/SFTP as well. (planned for June 2009)


Who is behind this?

  • Louis-Philippe Huberdeau
  • Marc Laporte
  • Louis-Martin Richard (tester)
  • Philippe Back (plans to add a nicer front-end GUI)
  • You? (We are looking for Beta-testers!!)


To do

Known Bugs

  • 2009-07-03 make instance on cvs BRANCH-1-9 doesn't detect
    • After manually entering BRANCH-1-9, make check is broken
  • 2009-04-13 latest branches/3.0 After a clean install via TRIM on php4-cli, I see "Your database requires an update to match the current TikiWiki version. Please proceed to the installer. Using Tiki with an incorrect database version usually provoke errors."
  • TRIM: don't kill templates_c/index.php
  • MacOS needs this path -> /opt/local/bin/php -d memory_limit=256M scripts/addinstance.php
  • On first backup, I get this error message "rm: cannot remove `/home/user/trim/trim/backup/1/manifest.txt': No such file or directory" (note: this has no side effects, warning only.)
  • Perform database setup... -> use stars for 2 password fields
  • my fedora has no php5 command - only a php command make instance can not be executed
  • On update, need space between instance name and URL
Which instances do you want to update?
[0] profiles.tikiwiki.orghttp://profiles.tikiwiki.org  lphuberdeau@tikiwiki.org




Roadmap

Phase 1
  • Disable site during upgrade (via .htaccess) Done
  • Update DB (1.x to 1.y sql script) Done
  • Handle both 1.9 and 1.10
  • Cron job for automatic check and report on file integrity (what about cache files?) Done
  • Something to indicate TRIM is working (sometimes, it is busy for many minutes and we are just left to wait, not knowing if it's stalled) ... may not be possible
    • Ok. Then, please indicate (in general) what TRIM is doing. Something like "TRIM will now transfer all files from SourceForge to TRIM (master) and then upload them to your instance. As long as you are not getting an error message, TRIM is hard at work! This operation could take 10-20 minutes or even a few hours, depending on server performance and network speed." Done
  • run fixperms.sh Done
  • Be more robust to various paths for php5 (ex.: using "which php" to find it) Done (using variables in Makefile so modification only required once)

Phase 2
  • For TRIM to handle backups Done
  • Make instance should suggest some values (ask for SSH host name and SSH user first). If you just type "Enter", it picks this value. These defaults should be editable somewhere in a config file. It also serves as a good example for the proper format. The default format which works for Fantastico hosting:
    • Web root : /home/$sshuser/public_html
    • Web URL : http://$sshhost (external link)
    • Working directory : /home/$sshuser/trim_working (TRIM should create with appropriate rights) Done
  • Admin interface to edit and delete instances (maybe a ssl protected web interface (external link)?) or here (external link)
  • To create database, database user and permissions from this user when installing Tiki. Could be in tiki-install.php as well
    • TRIM will need to be indicated the database login with sufficient rights to do this
    • Use this new info to populate db/local.php
    • Maybe pick profile from TRIM? (equivalent to tiki-install.php?) Done
  • Nicer error message (from .htaccess when site is being updated) Done
  • Backup
    • Cron job
    • Option to backup things outside of TikiWiki (ex.: mail directory, access logs, etc)
    • To be able to handle symlinks
      • /home/user/www is a symlink of /home/user/public_html and if I indicate /home/user/www as the web root of the instance, TRIM backup will not backup files) bug or feature enhancement request?
  • bundled _htaccess
    • Rename the bundled _htaccess to .htaccess for instances running on Apache
  • major upgrades
  • Easier to manage in shell interface
    • document so people know to type things like C 0-52
    • "Next page" when too much info in one screen (sometimes hundreds of lines of scrollback), but maybe this is a bad idea because it'll prevent unattended operations. hmmm
    • Add a note before entering a passphrase in addinstance.php : "If you enter a passphrase, you will need to enter it every time you run TRIM, and thus, automatic, unattended operations (like backups, file integrity checks, etc.) will not be possible."
    • If mysqladmin fails (ex.: bad login, not mysql but postgres, etc), it should offer to re-enter value or to abort database creation. In this last case, the system should indicate something like : "TRIM was unable to create your database with the information you provided. You should create it via your admin panel (cPanel, etc) and provide the information when you point your browser to tiki-install.php"
  • Indicate which warnings and error messages not to worry about (could be a link to a wiki page)
    • time stamp 2008-06-02 22:33:48 is 3938 s in the future
    • errors on setup.sh (when insufficient perms)
    • "warning: locate: warning: database /var/lib/slocate/slocate.db' is more than 8 days old" Nothing to do with TRIM
    • ssh: profiles.tikiwiki.org: Name or service not known Seems solved--
    • Make convert should handle 2.2 to 3.0 This is done by make upgrade


Phase 3
  • Should be a way to set a different folder/disk for backups (since this will become huge!)
  • TRIM should add db/lock file
  • TRIM should populate {$lastup}, which appears in templates/tiki-bot_bar.tpl. However, lastup should be (de)activatable and filterable by group.
    • Normal case is that I would activate and only let admins view it, but in some cases, Anonymous could see.
  • Set password at install (or anytime!). recreate admin use if deleted?
  • Investigate DB creation with cPanel
  • Once everything is nice and stable with SSH, add SFTP/FTP support (planned for June 2009) so we could use on hosts like Mosso Cloud sites (external link)
  • In addition to check and backup, having an automatic update script option (useful to always keep test sites like demo.tikiwiki.org up to date. Unattended updates won't be able to deal with SVN conflicts. Maybe just send a warning email?
  • Check used/available disk space. (% of used space) (Tiki misbehaves when disk is full and often we forget to check for that)
    • Report if less than x Megs left or x% (email warning)
    • Check on master and instances.
    • This seems unreliable on shared hosting

Phase 4
  • Backup
    • Have backups generate a "live" version of Tiki.
      • So a web agency could have a locally running backup of all their Tikis with yesterday's data.
        • Insures an easy way to test the automated backup system
        • Staff can continue to work if server or Internet connection is down.
        • Data could be in a UsbTiki format so Tiki admins could copy/sync to a mobile drive to have a backup in their pocket at all times.
      • Could also be to setup read-only mirrors
        • Tiki would need a read-only mode. So when users log in, they are informed that any changes here will be overwritten at the next sync.
        • These mirrors could be used for DNS3, DNS4, DNS5, etc.
    • Move an instance from one server to another (backup & restore in one step?)
      • This could include (This is the manual procedure I do) closing the old site ($site_closed eq 'y'), setting $site_closed_msg="This site has moved to: http://..." (external link) or "This site has moved to a new server. If you are seeing this message, it's because the DNS has not yet propagated for you. Please try again in a few hours or use this temporary address: http://..." (external link) before the backup and opening the site on the new server ($site_closed eq 'n').
    • Save to Amazon S3 (external link) and similar online storage services
    • Backup of the database should handle multitiki like doc/devtools/backupdb.sh
    • Investigate if scripts like automysqlbackup (external link), rsync-incr (external link) or rsnapshot (external link) have any concepts, features or code that would be useful for us.
  • Profiles
    • Investigate if/how TRIM could help maintaining: OpenSourceCMS type demo to test/develop and show off profiles
    • Hourly build of each profile: http://profiles.tikiwiki.org/profile_name (external link)
    • Investigate any synergy with the profile manager to apply database change to one, many or all instances.
      • Turn off unsafe, buggy or unused features
      • Add watch all wiki pages or watch all trackers to multiple Tikis
      • Use existing TikiWiki functions for mass operations. Ex.: add a TikiWiki user on dozens of TikiWikis, etc (In this case, it's important to use existing TikiWiki functions rather than alter the database directly)
  • Management
    • Add branch & lastup when listing instance.
      • Sort by branches & date, so as to see who is most out of date, who needs a security release, etc.
  • Install
    • At install, a maintenance.html (or install.html should be shown instead of files/folders)

Phase 5
  • Add a check for domain name expiry
    • This is useful because the person hosting the TikiWiki may not be listed in the DNS registries and won't be informed of DNS changes (expiry, domain theft, etc)
    • Some Tikis can have several domain names (domain parking)
    • Will this be easy provided the anti-spammer protections?
    • Warn by email (both to TRIM manager and instance contact)
      • 30 days before expiry
      • Ideally, on any change to the entry (for domain theft)
    • See script by Jean-François Pelletier, started at Codefest (external link)
  • Uptime/performance/content testing
    • Some of this, like spell checking and link testing, may be better done as a Tiki feature instead, so as to be able to access all content (web crawler will have problem with perms), and be usable without TRIM, but be "triggerable" via TRIM.
      • Check domain for presence of Google Analytics code
      • Check to see if Tiki is running properly (check presence of keyword like http://host-tracker.com/ (external link) and/or changes like http://www.changedetection.com/log/org/americana/index_log.html#changelist) (external link)
      • Check for the presence of a customized 404 apache error page (we often to forget to set this up)
      • Check response time
      • Check for broken links
      • Check spelling
      • Check missing images
      • Check for missing plugins (especially for non-bundled plugins which cause error when migrating text from site to another)
      • Check for standards compliance
  • Server monitoring
    • From future server admin panel
    • Interact with other tool (Nagios, etc.)


Phase 6
  • When suspicious files are detected, in addition to deleting them, it should be possible to send them to a non Web accessible "quarantine" for further analysis or even optionally send to the security squad.
  • Could enhancements to _htaccess be merged to .htaccess? (ex.: rename .htaccess to _htaccess just before svn/cvs up) so if I have a modified it, I still benefit from community enhancements.
  • Test/document on Cygwin (external link) so users could manage remote TikiWikis via their Windows PC or Mac. Or a VMware appliance (external link)
  • Initially built for Linux-Apache-MySQL-PHP (LAMP). Later, we will adjust TRIM to work with different setups.
  • Investigate a TRIM-like application that works directly on the remote instance. A small php script which fetches the latest version of Tiki and installs it. (Could be impossible because of limited file permissions that php scripts, with Apache rights, that have limited permissions). mods have this issue.
  • import mysql data (external link) a bit like BigDump (external link)
  • Possibly automated running of testing feature.
  • Investigate if we should make additional use of .htaccess in various directories (only useful for Apache)



Wishlist

 RatingSubjectPriorityInvert SortReport typeVersionFeatureCreated
open2/2OpenSourceCMS type demo to test/develop and show off profiles8Community projects
Feature request
2.x
Dogfood on a *.tikiwiki.org site
Installer (profiles, upgrades and server-related issues)
Profile Manager
TRIM
2008-01
pending2/2templates_c cleaning is too aggressive and deletes system files8Bug: Error1.9.x
2.x
Administration
Templates (Smarty)
TRIM
2008-05
open2/2Apply for SourceForge hosted Apps8Community projects
Feature request
3.xMultiTiki
TRIM
2008-11
open2/2Round Robin / Redundancy / Disaster planning for all *.tikiwiki.org content6Community projects
Feature request
Dogfood on a *.tikiwiki.org siteBackup
TRIM
2007-07
open - Setting admin password in the installer, with option to force change at first login6Feature request4.xInstaller (profiles, upgrades and server-related issues)
Security
TRIM
2009-05
closed2/2Secdb automatic check with cron job5Feature request1.9.x
2.x
Administration
Installer (profiles, upgrades and server-related issues)
Security
TRIM
2007-09
open - tiki-install.php to create mysql user, mysql database and assign permissions1 lowFeature request2.xInstaller (profiles, upgrades and server-related issues)
TRIM
2008-03


Contributors to this page: marclaporte21258 points  , sylvie2073 points  , lawiz17 points  and lphuberdeau580 points  .
Page last modified on Thursday 02 July, 2009 21:07:04 UTC by marclaporte21258 points .

To register

To have an account at this site, please register at Tikiwiki.org (external link), and then use that user name and password to log in here.

This site gets user information from Tikiwiki.org with the InterTiki feature.

Last Comments

  1. more information from sylvie
  2. can not reproduce
  3. i could repeat it
  4. fixed in 2.4 thanks!
  5. seems to work for me

Search a Wiki Page

Exact match

Search Tracker Items Subject

Keywords

The following is a list of keywords that should serve as hubs for navigation within the Tiki development and should correspond to documentation keywords.

Each feature in Tiki has a wiki page which regroups all the bugs, requests for enhancements, etc. It is somewhat a form of wiki-based project management. You can also express your interest in a feature by adding it to your profile. You can also try out the Dynamic filter.

Accessibility (WAI – 508)
Action log 2.x
Administration
Ajax 2.x
Alert 3.x
Articles & Submissions
Backlinks
Backup
Banner
Blog
Bookmark
Browser Compatibility
Cache
Calendar
Category
Charts
Chat
Comment
Communication Center
Compression (gzip)
Consistency
Contacts Address book
Contact us
Content template
Contribution 2.x
Cookie
Copyright
Custom Home (and Group Home Page)
Database independence
Database MySQL
Date and Time
Debugger Console
Directory (of hyperlinks)
Documentation link from Tiki to doc.tikiwiki.org (Help System)
DogFood
Drawing
Dynamic Content
Dynamic Variable
Ephemeride
External Authentication
FAQ
Featured links
File Gallery
Forum
Friendship Network (Community)
Game
Gmap Google maps
Group
Help System
Hotword
HTML Page
i18n (Multilingual, l10n, Babelfish)
Image Gallery
Import-Export
Install
Integrator
Interaction
Inter-User Messages
InterTiki
JS Calendar
Karma
Live Support
Lost edit protection
Magic 3.x
Mail-in
Map with Mapserver
Menu
Meta Tag
Missing features
MindMap 3.x
Mobile Tiki and Voice Tiki
Mods
Module
Mootools 2.x
Multimedia 2.x
MultiTiki
MyTiki
Newsletter
Newsreader
Notepad
OS independence (Non-Linux, Windows/IIS, Mac, BSD)
PDF
Performance Speed / Load
Permission
Poll
Profile Manager
Quicktags
Quiz
Rating
Registration
RSS
Score
Search engine optimization (SEO)
Search
Security
Semantic links
Shoutbox
Site Identity
Slideshow
Smarty Template
Smiley
Spam protection (Anti-bot CATPCHA)
Spellcheck
Spreadsheet
Staging and Approval
Stats
Survey
System log
Tags 2.x
Task
Tell a Friend + Social Bookmarking 2.x
TikiTests 2.x
Theme
Trackers
Trackers (Mirror)
TRIM
User Administration
User Files
User Menu
Watch
WebHelp
Webmail and Groupmail
WebServices 3.x
Wiki 3D
Wiki History, page rename, etc
Wiki plugins extends basic syntax
Wiki syntax text area, parser, etc
Wiki structure (book and table of content)
Workflow (Galaxia Workflow engine)
Workspace
WYSIWYCA
WYSIWYG 2.x
XMLRPC