updates

I've been doing one or two pages starting from 1AD and working towards the present. I was finding going through the giant influx of new information for the more modern timelines tedious, I'd have to do a large block at a time to make one page look good, and I found I was slacking off and not doing it the way it should be done. So I'm doing about a page a day, editing and removing duplicate lines. When I get in 'the mood' I'll work on the more modern lines, which involves more support work like building the history directory up and finding cyclopedia articles.

I've also did a little work on the donation ad reduction. I've gotten it to work fairly well on teh side menu items, and now I have to tackle the top and bottom banners and the few pop-up ads I have.
Once all that is completed I'll build a page to keep track of donations and the goal and explain what the purpose of the whole thing is.

donations vs. advertising

I'm going to try something to reduce the amount of advertising I need to do to keep this site running in its present (growing) form. I've added a paypal donation button to the side bar here and there are some scattered throughout the timelines site. What I propose is that I will set a goal amount, which will be the average amount of revenue I get from affiliate advertising and each month as doanations approach that goal I'll reduce the amount of affiliate advertising accordingly the following month. So if The monthly goal is set to $20 and I get $10 in donations I'll cut the affiliate advertising in half. That will be top and bottom banners and most items in the menu bars on the timeline pages.

How the mechanics will work is that I will have 'dial' that can be set on the template program that will determine tha amount of advertising to show as I update individual pages. As the donations come in for a given month I'll turn this dial up as I'm updating. So if i get a donation for $1 on the dial will be set to 1/20th and all pages will have that reduction. Then, if a couple days later another donation comes in for an additional $5 the setting will be at 6/20th. This doesn't seem particularly even because not all the pages will be changed, but which ever changes are changed will remain that way until they are updated again. So it will be slow to reduce slow to raise. At the end of the month the setting will be brought back to 0/20th.

That is the current plan. I will be working with paypal to get the information worked out to make reporting automatic either in this side menu or another active place on the site.

other fixes

To go with the new page types I had to change the template program that finds the correct date links. I worried over it and stared at it for a while before attempting it, but it worked out really well for the quarter pages.

Encouraged by this I went ahead and built 5 pages for 1948 1948 Events and up. They link into 1949 and to the existing older 1940s pages below.

Now I'm behind on editing the way I wanted to, I've found myself bouncing around doing little edits all over. One problem was with the parsing program that rebuilt the database, It made some (read 'the programmer') errors when deciding which inline database item went with the inline link. Most are not off by much, but they really can't be ignored and have to be fixed by hand.

Another fix was the darn side menu advertising. I have a love hate relationships with both the way it works and the need to have it. I found I had made some mistakes in some if statements which were compounded by some copy and pasting. They are fixed and pages built from this point on will have the full range of advertising. What I like about the system is it makes thing varied and automatic, what I don't like is having to have it in general and the system is a little confusing at times, since some ads can't be placed with other ads.

I need to have it though to help pay for hosting and at least some of my time. I was setting up a system with paypal that I'd reduce advertisng if I got donations, based on a preset goal, if I got the full goal then i'd remove all advertisng for the following month of updates. but it never got off the ground. I may rethink it and make a more coherent system around the concept.

new feature

I'm testing something in the BCE timelines. When There is no registry directory or other links out for a given inline link term then a web search link will appear witht he link. This will make a searcch through Banner Boxes affiliate link. They are in turn owned by Rolist, who gets its backfill after paid links from inktomi.

These links are presented to help you move on if there is nothing on the timelines site that catches your interest.

fresh pages

I'm uploading the second set of finished pages, the data in them is still very raw. As I go on I'll be putting together a DB editor just for the database to make things easier.

With the bulk of information now I'm dividing up pages more. Many years will be divided into quarters or even months and each year page will have a corresponding 'events' page, which will have the items that do not have detailed month/day dates in them.

I've uploaded 5 new pages centered on 1949. the 1949 Events and
1949 Q1 1949 Q2 1949 Q3 1949 Q4.

These pages are not hooked directly into the rest of the timelines yet, I'm going to wait for the natural course of uploads and editing take care of interpage linking on the site, then I'll remove the remaining unlinked pages .. as you'll notice the 1949_timeline.shtml is no longer in the link menu of the new pages.

I'll be categorizing and inline linking these 5 pages before going on to the 1948 set .. so it will be a long process, but the end result should be very satisfying.

maybe the last leg

I'm nearing the end of processing the rough database and tweaking the template builder. I've built test version of nearly every page type I plan on using and have firm plans on doing the others.

The database is much larger than anything I've dealt with before, It will be an ongoing process to edit and make other corrections. It will also be an interesting challenge to make sure everything is presented well.

first upload

I've uploaded the first set of pages built using the new system. 5000 to 3000 BCE up to the 599 to 550 BCE timelines.

This test run found two seperate errors, I had the parser looking for 2 inline date link styles when in fact there are 4. The other was the start date for the BCE pages were off by one (I had known of the later problem in a cursory way, but had conveniently forgotten about it.)

I choose the BCE liness to start testing because they had the least amount of new information in the database and generally I like to start at the beginning. Also, I'm still working on some of the features found in later timelines, like the single year lines and single year event pages.

templating and other things

I built a one page template, with page specific items stored in the database. This will make it much easier when editing the timeline pages. I'm going to build a few more basic templates to handle other page types, like the era indices and some of the static pages.

It should be noted that while much of what presented here is automated it is not automatic. I still build the pages at home and each page fits into a determined structure. The templates and database tools I've created are just tools to this end. All the pages except for a few SSI are static. I do it this way to keep my load on the server and the load times for visitors as low as possible, and it is done because all of this is homemade by me and I really wouldn't trust it 'in the wild' without me looking at what it is doing.

The size of the 'working' database is about 20,000 items. There are still plenty of dupes and no specific editor yet. I'm still writing throwaway scripts to extract and convert the different timelines pages into the mySQL . I think I have only one more set to do.

I'm still learning the intricacies of mySQL, but I have a decent idea of what the timeline tools will look like when all this is finished and how to get there.

databasing update

the database is growing again. There was a side project I had worked on a little bit called Today in History. In those pages is much of the data I had lost in my XML acciedent. So I'm going through that data, removing dupes and adding it to the current database.

This is not an exact science, There are small errors in places. I hope to be presenting pages with this updated dataset in the next couple of days. along with the templates reformatting I hope to be up to the end of the 1500s also.

The History of Din Timelines Backend

Here is the big picture. At its inception timelines had basic html pages, each page with the items directly on it, including all the links in the text (inline links) which were hand written. Next came a perl/ssi version that lasted several months, each inline link was changed to an SSI in the shtml pages that called a little perl script and an flatfile database. This worked fine until my host gave me a friendly CPU usage warning, which was caused by the banner script I was running, but I thought it best to change all those perl calls.

This brings us to the stage where I wrote a perl script to run at home, revamped the database structure, added an editor and did all sorts of fancy things. I had templates of all the pages, each page with all its items with markers for the inline links in them. Id run the script and it would run the templates through the parser and replace the markers with the links and then I would upload these finished pages. Additions to this included building a system to place ads in the side menu based on the page size.

Then two years ago I had a major harddrive crash and lost all of everything. So i found a newer version of hte home server I had, with php and started rewriting everything in php. it wasn't long before I installed mySQL and started playing with it. placing the inline links in mySQL and the actual timelines in an XML file that would be parsed into the template, which is now just formatting without any information except for what information needs to go into it.

From this point on everything will be in mySQL except for the smaller flatfile DBs with the navigation information between timelines. In the future I;m looking at further abstraction between content and formatting.

first database tests complete

After a few bugs and a little hunting around I've rebuilt the active database back to about what it was. Which puts me back to the point where I was. Write now 5000 BC to 1599 is in the new database and seems to be working fine, I've built various pages and things look good.

This buts me back into the schedule where I was slowly beefing up and rewriting the old system.

Starting to Remake Timelines

After some testing last night and this morning I'm fairly confident that the new database is viable enough to start redoing and uploading pages, there is much information that I have double copys of mixed in various back-up and information that was lost. The BCE information has the lesser amount of redundance so that is what I'm going to try and work through today.

THe vague plan is for me to go dump information into the mySQL one 'page' at a time. Then rebuild the page from the database compare the two pages then upload the newer and go to the next page. A good readl of my time will be spent looking for other back-ups and checking them against the current.

I hadn't written anything into the database since my last major upload, so I feel confident that everything up to 1400 will be handled this way. from that point on i have two sets of information up until sometime in the 1800s with everything in the old templates. It will be a long tedious process.

My todo list for this project is to make a editor specific to my needs, I've a few DB editors but the don;t easily have some fo the functions I need.

Another issue is speed. I'm thinking strongly of reducing image advertising a bit, and stick with lighter stuff, and less of that too.

New Database

After about 8 hours of typing php (4 last night and 4 this morning) I made a parser that took the existing timeline pages and made them into a mySQL insert file so I can manually review it .. big regexp can be big problem .. Then I changed the parser that turns the timelines templates into the pages on this site to work from this new database table. As things stand now the new database, with only text data for about 50 items, is working.

Things that /may/ work are the BC dates, I rewrote parts pertaining to going backwards, I'm pretty sure it will work, after I test and fix the inevatable errors it will cause.

The finished database, which never got exposure on timelines will be about 20,000 individual items. I have to rewrite a few things to take what is left of my xml databases to include them in I can make phpmyadmin import them but I'd feel better writing something myself .. after this long mess it wouldn't take very long.

There is a hunk of data that I've lost, things that were in the main XML file that were not on the site nor in the original sub XML files.

On the much suckier side, I've got to go to work for 8 tonight at the store then 8 again in the morning. It is at these times I regret getting a real job for exercise so I'm not sitting in front of the computer all the time.

Loss of Data

In the midst of the long clean-up at timelines I had a hiccup on my home machine and lost the entire database i use to build the timeline pages. I'm working out a way to rebuild from the extant finished pages even though about half of them never made it into the new database to begin with.

I can automate most of the process, but there are a few things I will have to do by hand. like setting some of the link titles that go to un-pretty database names. An example is 'ireland_1700' is a database name but I set the title simple as 'Ireland' because the ireland-1700 is only used on the .. 1700 pages. Another example is when a person title changes or different version sof the name are uses, but still goes to the same item in the database .. Emperor Napoleon, Napoleon Bonaparte, Napoloeon, etc. all point to the same database name.

So some other projects I have will be put on hold while this goes on, I believe I will be putting everything in the same mySQL database I'm using to store the inline links information .. It should make things easier in the future, even though I have only a small experience with mySQL.

Another Move

I pulled the simply awful webmaster pages from timelines, made them slightly less awful and moved them to here. The old webmaster pages have a link to the new ones, I'll change internal linking as other things are updated.

Purge Begins

Earlier today I re-instituted Site search through SiteLevel. I had used them before, the last time around jan 2001. But indexing with Fluid Dynamics was not getting done, I seldom had time to site through the process and I was always worried about server load and CPU cycles when I was doing it. To be fair neither of those issues were ever a problem with FD, which is a fine program, but more than I can work with right now.

So I'm going to uninstall the old site search. I've already got the site level linked up and working in tandem with my nifty search redirect program.

Likewise I believe I'm going to terminate the old forums and some other supprt pages that would be better served on other domains I operate. Some I have planned to move for a while, other are just collecting dust.

Cheap and Easy

I took out the main page that led to the forums and changed it to lead to this weblog. The forums are still up, but not readily accessable from the outside. I will more than likely just delete them at a later date.

Welcome to Test

We will have all your test.

After some debate I decided to close the timelines forums. They were not doing what I wanted them to do for a variety of reasons, mostly because I didn't have the time to devote to them. I've decided instead to switch focus over to this weblog. It will also serve as the replacement of my vaguely lamented newsletter.

I will be able to handle both from this system a little bit better without taking all the time out to load up seperate systems and such.

The intention will be to allow people to send me questions or information via My Contact Page. Which I will either reply to directly and/or post here to allow comment.

The time of intergration is unknown, there are hundreds of pages at D.Timlines. I will do rudementary linking today.