23000 Images off Photobucket into SacredWiki!

[gogoblender 3,068]

As well as the countless other things Schot's been running around on the forum about lately, one of our latest fires has been the SacredWiki Photobucket running out of time in ... what three days? lol some members here had sent pms and the downloading process off the bucket database has been going all day. We apologize for the lag, I don't think anyone would have ever guessed how many images there were coming off the bucket dbase. And lately bucket has been a horror to work with. Insane log in times, lag, and huge time for just making clicks. With the wiki's entire image dbase now moved to our home servers, we can start using the actual built in Wiki method f directly uploading to the wiki's own image file storage. We're finally going to get off the leash of the two photobuckets we had to use to keep hosting images for the wiki and DarkMatters and finally have some control and added speed!

 

 

gogo

[wolfie2kX 528]

ACK! 23,000 pics! So does this mean we're going to have to fix all of the pic links in the Wiki now? Or is this going to be done automatically somehow?

[dreeft 9]

ppft, only 23 thousand?

[Schot 407]

And 1! ^^ It's that extra 1 that really makes it big huh. Haha. It's pretty incredible just how many pics we've amassed. I'm sure a good 10% of them are no longer of use but still quite a lot are active. The whole account had a single focused goal in mind which means barely any of the pics are merely random "fun pics". It's about 980 MB if I recall. The bucket served us well but it's high time we moved onwards and upwards. Pretty exciting!

 

 

With any luck I'll run a successful Search & Replace request on our wiki database. One of the perks with a Photobucket Pro account is that it provides ftp access so I was able to download our entire collection with folder order intact and then upload it to our server. All that's left is to run a command that will search the wiki for the bolded part of

http://i241.photobucket.com/albums/ff165/SacredWiki/Wiki%20Layout%20Images/MainPage/MainSacredsmall.jpg

in all links and replace them with

http://www.sacredwiki.org/images

 

I think I'll make a backup first though... ^^

[Dragon Brother 619]

And 1! ^^ It's that extra 1 that really makes it big huh. Haha. It's pretty incredible just how many pics we've amassed. I'm sure a good 10% of them are no longer of use but still quite a lot are active. The whole account had a single focused goal in mind which means barely any of the pics are merely random "fun pics". It's about 980 MB if I recall. The bucket served us well but it's high time we moved onwards and upwards. Pretty exciting!

 

 

With any luck I'll run a successful Search & Replace request on our wiki database. One of the perks with a Photobucket Pro account is that it provides ftp access so I was able to download our entire collection with folder order intact and then upload it to our server. All that's left is to run a command that will search the wiki for the bolded part of

http://i241.photobucket.com/albums/ff165/SacredWiki/Wiki%20Layout%20Images/MainPage/MainSacredsmall.jpg

in all links and replace them with

http://www.sacredwiki.org/images

 

I think I'll make a backup first though... ^^

Good luck with that..hopefully it works as planned

[wolfie2kX 528]

With any luck I'll run a successful Search & Replace request on our wiki database. One of the perks with a Photobucket Pro account is that it provides ftp access so I was able to download our entire collection with folder order intact and then upload it to our server. All that's left is to run a command that will search the wiki for the bolded part of

http://i241.photobucket.com/albums/ff165/SacredWiki/Wiki%20Layout%20Images/MainPage/MainSacredsmall.jpg

in all links and replace them with

http://www.sacredwiki.org/images

 

I think I'll make a backup first though... ^^

 

Very cool - and yes.. VERY good idea to do a backup first..!

[Schot 407]

Thanx Dragon!

 

Haha wolfie. Must protect precious pics eh. ^^

 

 

 

Just for fun I made a manual swap of some pics which worked perfectly and exactly as I mentioned above. Demonic Being is now using the image versions from our server. All I did in the edit window was swap this:

http://i241.photobucket.com/albums/ff165/SacredWiki/Wiki%20Layout%20Images/MainPage/MainSacredsmall.jpg

for this:

http://www.sacredwiki.org/images

 

Give it a try if you like!

 

 

P.s.

I guess we should stop any uploads to the Photobucket from now on. I think it's a done deal. I'll probably run the search and replace command tomorrow.

[dreeft 9]

I'm getting a 403 Forbidden on the main image on the Demonic Being page.

 

This one: http://www.sacredwiki.org/images/Sacred_2/Bestiary/Demon-Quest/Dem-DemonBeing.jpg

 

Looks like a fairly simple permissions error, just thought I'd point it out

[Schot 407]

Hmmm.... I'm not seeing that. It looks ok on my end. From that url our permissions shouldn't be different. Gonna have to tinker a bit. Does it resolve if you do a hard refresh?

[dreeft 9]

It wasn't working on any of my browsers on my main computer, but was working on my 2nd (and 3rd) machines without any problems.

 

Must have just been taking it's time to get there, or I opened it too quickly and it cached without, seems fine now

[Schot 407]

Ah, whew! That's good news. Was beginning to worry and I haven't even touched the database yet, lol. Thanx for testing dreeft! It's much appreciated.

[wolfie2kX 528]

Thanx Dragon!

 

Haha wolfie. Must protect precious pics eh. ^^

 

 

 

Just for fun I made a manual swap of some pics which worked perfectly and exactly as I mentioned above. Demonic Being is now using the image versions from our server. All I did in the edit window was swap this:

http://i241.photobucket.com/albums/ff165/SacredWiki/Wiki%20Layout%20Images/MainPage/MainSacredsmall.jpg

for this:

http://www.sacredwiki.org/images

 

Give it a try if you like!

 

 

P.s.

I guess we should stop any uploads to the Photobucket from now on. I think it's a done deal. I'll probably run the search and replace command tomorrow.

 

Very cool... Fixed a link on that Demonic Being page - I *KNOW* I did the Arena in the Clouds chain quest... looks good. So now that we're no longer putting stuff up on Photobucket, how/where do we put the pictures now? Anything special we need to know?

[Knuckles 904]

Very cool Schot. The loading times for photobucket were getting pretty long. I've been uploading 8-9MB per 'Places' page in pics

 

So it's ok to start uploading directly to sacred wiki now? Also let me know if any images are broken (Was using [url/img] tags quite alot in the places section (had a hard time getting pics to display correctly without using them)

[Schot 407]

Thanx Dragon!

 

Haha wolfie. Must protect precious pics eh. ^^

 

 

 

Just for fun I made a manual swap of some pics which worked perfectly and exactly as I mentioned above. Demonic Being is now using the image versions from our server. All I did in the edit window was swap this:

http://i241.photobucket.com/albums/ff165/SacredWiki/Wiki%20Layout%20Images/MainPage/MainSacredsmall.jpg

for this:

http://www.sacredwiki.org/images

 

Give it a try if you like!

 

 

P.s.

I guess we should stop any uploads to the Photobucket from now on. I think it's a done deal. I'll probably run the search and replace command tomorrow.

 

Very cool... Fixed a link on that Demonic Being page - I *KNOW* I did the Arena in the Clouds chain quest... looks good. So now that we're no longer putting stuff up on Photobucket, how/where do we put the pictures now? Anything special we need to know?

 

 

Cheers for the fix wolfie. Your link worked out nicely.

 

Very cool Schot. The loading times for photobucket were getting pretty long. I've been uploading 8-9MB per 'Places' page in pics

 

So it's ok to start uploading directly to sacred wiki now? Also let me know if any images are broken (Was using [url/img] tags quite alot in the places section (had a hard time getting pics to display correctly without using them)

 

I'll have an update on how we're going to upload soon but I don't think we'll be doing any uploads today. There's still a lot of fixing to do yet. We're getting there and should be a lot easier than that laggy old bucket once we're set.

[daemonrocks23 17]

I know that on wikipedia you go to a "Special:" page which is Special:Upload, it should be the same for all wiki's. I set up a wiki once, and I switched a number 0 to 1 and *BAM* it was working like a charm!

[Schot 407]

Very true daemonrocks. Good point. All we gotta do now is figure out a way to grant permission to everyone to upload. Unfortunately everyone's upload rights are off at the moment. Our bridge used to manage that kind of permission for us and we had a convenient interface to make adjustments. Now that we've updated that function is no longer working so I'm going to have to learn how to use mediawiki's user rights controller. Hopefully the bridge won't interfere with that. We'll see. ^^

 

 

First sweep was a success! After trying out about 15 variations of the query I wanted to use I finally pegged it and the sweep ran. I didn't want to do anything major so a created a mini sweep that would correct the photobucket thumbnails that when clicked on would load the larger image version of that thumbnail within the Photobucket website. The thumbnails at the top of the Fan Art page were linking to photobucket. The query I ran corrected those and in total made 1537 changes. That could be either pages or images but I'm guessing it was number of images.

 

And here's the handy line of text that made it all happen:

UPDATE `content` SET `old_content` = REPLACE(`old_content`, '?action=view&current=', '');

 

At first I just assumed that I would only need to run one query but it turns out I need to run 5. The next step now is change all the photobucket images into sacredwiki images.

 

UPDATE `content` SET `old_content` = REPLACE(`old_content`, 'http://i241.photobucket.com/albums/ff165/SacredWiki', 'http://www.sacredwiki.org/images');

 

Once that's done I'm off to change all the thumbnails from bbcode into wiki code. This one is a 3 parter since there are 3 parts to the code that makes up a bbcode thumbnail.

[url=[/html]

 
[sql]UPDATE `content` SET `old_content` = REPLACE(`old_content`, '[url=', '[');[/sql]

 
[html]][img]

 

UPDATE `content` SET `old_content` = REPLACE(`old_content`, '][img]', ' ');

 

[/img][/url]

 

UPDATE `content` SET `old_content` = REPLACE(`old_content`, '[/img][/url]', ']');

 

Reason being is that the bridge we are using to share database info between the wiki and the forum here is no longer supported and very outdated. That bridge is what is allowing us to use bbcode on the wiki and once it's back finally breaks we'll be in for a nasty surprise. The author of our bridge has in fact been awol for about 6 or 7 months now... Scary huh!

 

Well... Here goes nothing!

[Dragon Brother 619]

fingers crossed!

[gogoblender 3,068]

It's pretty cool actually. It's like he's setting off these controlled burns or mutatations that fire across the dbase... this last one's gonna be HUGE

 

Go mighty Myst Master, thy time has cometh!~

 

 

gogo

[Schot 407]

Woohoo! *snatches Potters' wand and pushes him aside* There can be only one!

 

 

Ok, so that didn't go quite as planned. I just ran the second sweep. "22256 row(s) affected. ( Query took 2.5163 sec )" After 22,256 changes the images are still pointing to Photobucket. As it turns out I only swept the history, haha. Now I gotta figure out where the database keeps current page info...

[Dragon Brother 619]

haha...thats rather amusing, probably not so much for you though.

[gogoblender 3,068]

heh, it is kind of fun though. We're getting to wave our hands about and make whooshing noises as Schot points blasts across the DB in search of the images.

 

 

 

 

 

gogo

[Knuckles 904]

I'D offer to help but I only know enough to be dangerous. I can only imagine the havoc I'd unleash in there if I tried something like that. But I have total faith in Schot taking care of all this, it's either that or I'm manually editing 1000's of pages

 

 

Free donuts on the other side if you're successful Schot !

[gogoblender 3,068]

I hear a happy Whoosher running around today ... I think our forum wizard has succeeded in his task...

 

Blast ye all back to dee bucket!

 

 

 

 

 

gogo

[Knuckles 904]

Uh oh...gogo's gone techno-magical on us again

[Schot 407]

All done! Whew!

 

Gentlefolk... Sacredwiki has kicked the bucket! In a good way. Photobucket provided an alternative image hodting resource for several years and if not for it we may not have grown our image library so quickly. Our needs have changed though and SacredWiki has achieved enough power to start taking care of business on it's own. Thanx to all who have helped it become the wonderful wealth of knowledge it is today.

 

As of now all of our 23,000+ images are hosted and linking to the SacredWiki server. However! There's a few images out there on SacredWiki still hosted from Photobucket. They are Photobucket images hosted from member accounts as opposed to our SacredWiki photobucket. Since those images are not hosted on the sacredwiki server I couldn't sweep them out so we'll need to finish off the rest manually. I'm not sure but there's probably only a couple hundred. Maybe less... First though I need to get the upload function working for everyone which is proving tricky. The Forum/Wiki bridge I mentioned before is preventing me access to grant permission to upload for members so I'm digging through the database once again to try to commandeer control over permissions. It's proving a challenge to say the least, haha.

 

That's all for now! Hopefully I'll have an update very soon.

 

P.s.

Here's the low down on what I did on the database:

UPDATE `content` SET `old_content` = REPLACE(`old_content`, 'http://i241.photobucket.com/albums/ff165/SacredWiki', 'http://www.sacredwiki.org/images');

22256 row(s) affected. ( Query took 2.5163 sec )

 

UPDATE `text` SET `old_id` = REPLACE(`old_id`, 'http://s241.photobucket.com/albums/ff165/SacredWiki', 'http://www.sacredwiki.org/images')

4290 row(s) affected. ( Query took 0.5677 sec )

 

UPDATE `content` SET `old_content` = REPLACE(`old_content`,'?action=view&current=','');

1537 row(s) affected. ( Query took 0.4367 sec )

 

UPDATE `content` SET `old_content` = REPLACE(`old_content`,'

10170 row(s) affected. ( Query took 1.6317 sec )

 

UPDATE `content` SET `old_content` = REPLACE(`old_content`,']

10141 row(s) affected. ( Query took 0.5564 sec )

 

UPDATE `content` SET `old_content` = REPLACE(`old_content`,'' alt='>UPDATE `content` SET `old_content` = REPLACE(`old_content`,''>',']');

10134 row(s) affected. ( Query took 0.5987 sec )

 

 

So let's see... That's about 58,528 edits I guess we could say. I think I reached my quota for the year, haha. ^^