Linux partition format

for all subjects/topics not covered by the other forum categories
Post Reply
User avatar
BeebMaster
Posts: 2615
Joined: Sun Aug 02, 2009 4:59 pm
Location: Lost in the BeebVault!
Contact:

Linux partition format

Post by BeebMaster » Mon Oct 08, 2018 12:24 pm

There's a problem with the hard drive in my laptop, it seems to be dying on me. The disc utility tells me the disc is "OK" but has 1160 bad sectors. It was only 784 a couple of weeks back, but since then it has been a case of applications disappearing before my very eyes, followed by spurious errors and eventually freezing.

It is only about 18 months old, but it has jumped off tables etc. a few times, and has been in at least 2 aeroplanes, so maybe it isn't surprising.

I've used a Live CD USB stick to fsck all the partitions but it is still having problems.

When I set it up I made several different partitions - ext4 system partition, ext4 home partition, ntfs partition for use with various Virtual boxes, and a bit of swap at the end. (I think having a separate syst and home partition is supposed to be the recommended setup, but I'd only ever had a single ext4 partition the whole size of the disc before).

It would be a major faff to effectively start again from scratch with this disc in terms of all the setup and installation required. I've just about got Ubuntu where I want it, after 10 years, even with 18.04.

I'm guessing that the problems are mainly in the system partition since apps are crashing and what not, so is it possible to properly format just this partition? In fact is it possible to properly format (in the old world sense we understand on here) a disc at all nowadays?

I'm thinking I could make an image of the system partition (it's about 128GB) with dd and then restore it after format. Potentially I could make an image of the whole lot, I should have an external drive big enough to do it (1000GB) - but that would take a very long time.

Before you all start on me - I did determine when I bought this machine in March 2017 that it would be my last new computer with a mechanical hard drive, the next one will be SSD!
Image

Prime
Posts: 2579
Joined: Sun May 31, 2009 11:52 pm
Contact:

Re: Linux partition format

Post by Prime » Mon Oct 08, 2018 3:17 pm

One piece of advice if you haven't already, backup your data now!

Bad sectors could after all be real bad e.g. failing sectors, to me it sounds like you need to replace the disk. If the internal spinny disk is SATA, if it's only 18 months old it will be....then replace it with an SSD, it's a pretty simple swap (depending on how hard it is to actually get the laptop apart to the point where you can change the disk!).

Bonus being that it will seem like a new machine :)

There are better solutions to copying the data than dd, clonezilla for example will copy only the sectors that it needs to, the ones that the filesystem is actually using, so tends to be much faster. ddrescue may also be useful if the filesystem is corrupt, I've used it a few times to recover data from failing disks, not just PC ones either, it will work with raw sectors so in one case I used it to dump a (failing) Amiga hard disk on a PC.

Cheers.

Phill.

Coeus
Posts: 1079
Joined: Mon Jul 25, 2016 11:05 am
Contact:

Re: Linux partition format

Post by Coeus » Mon Oct 08, 2018 4:39 pm

BeebMaster wrote:
Mon Oct 08, 2018 12:24 pm
...has 1160 bad sectors. It was only 784 a couple of weeks back...
In my experience the rate of new bad sectors appearing will continue to increase fast. I suspect what is going on that there has been a minor head crash which has scraped some material off the disc surface and this is now floating around inside the enclosure. Each time a piece goes between the head and the surface it breaches the tiny gap between them and scrapes a bit more off, thus rendering another few sectors unusable and further increasing the amount of debris floating around in the enclosure and shortening the time before it happens again. As time goes by the rate of damage will accelerate.

So best to get as much data off it as possible as soon as possible. Depending on how much space you have on another system, your priority should be your /home partition, assuming that is where you data is. Regarding the OS, if you can at least copy /etc to somewhere else and get a list of the packages installed - the package manager should be able to give you a list - then you should be able to turn a fresh install into an OS configured as you like it reasonably easily.

Coeus
Posts: 1079
Joined: Mon Jul 25, 2016 11:05 am
Contact:

Re: Linux partition format

Post by Coeus » Mon Oct 08, 2018 4:42 pm

I wrote:
Coeus wrote:
Mon Oct 08, 2018 4:39 pm
and get a list of the packages installed - the package manager should be able to give you a list
See this: https://askubuntu.com/questions/9135/ho ... d-packages

User avatar
BeebMaster
Posts: 2615
Joined: Sun Aug 02, 2009 4:59 pm
Location: Lost in the BeebVault!
Contact:

Re: Linux partition format

Post by BeebMaster » Mon Oct 08, 2018 7:02 pm

Thanks for the advice, it's the loss of a working computer set up the way I like it more than anything which will be missed if it crashes. I don't think I'd lose any data, it's all stored separately and accessed as an NFS share running on a Raspberry Pi 2. I have 3 backups of that, including one off-site in my storage place, so I hope that's covered.

Maybe I will try to replace the hard drive with an SSD one and backup the current drive using the methods suggested. Brings back memories of having to replace the hard drive in my original Olivetti laptop (some of you saw it at Wakefield 2013) in February 1999 when the original drive died containing my dissertation! Luckily I had a backup on 1.44MB floppy!

Getting a list of installed packages is a good idea, I've been keeping a record of system "changes" since I started the computer in March 2017. It isn't exhaustive but the list runs to 69 steps, so I wouldn't really want to have to begin again from a fresh install unless a last resort.
Image

User avatar
BeebMaster
Posts: 2615
Joined: Sun Aug 02, 2009 4:59 pm
Location: Lost in the BeebVault!
Contact:

Re: Linux partition format

Post by BeebMaster » Mon Oct 08, 2018 7:05 pm

Actually - thinking about it, I don't know where the VBox virtual drive files are stored, I don't have a backup of them, or anything on the NTFS partition. Better get onto that quick.
Image

User avatar
richardtoohey
Posts: 3590
Joined: Thu Dec 29, 2011 5:13 am
Location: Tauranga, New Zealand
Contact:

Re: Linux partition format

Post by richardtoohey » Tue Oct 09, 2018 4:41 am

Prime wrote:
Mon Oct 08, 2018 3:17 pm
Bad sectors could after all be real bad e.g. failing sectors, to me it sounds like you need to replace the disk ... replace it with an SSD
+1 to all that. From my experience it's not a good idea to tolerate bad sectors - it's a real warning that the drive is going to fail. You might get away with things for a while, but it's only going to end one way ... :cry:

And SSDs =D>

User avatar
BeebMaster
Posts: 2615
Joined: Sun Aug 02, 2009 4:59 pm
Location: Lost in the BeebVault!
Contact:

Re: Linux partition format

Post by BeebMaster » Tue Oct 09, 2018 7:43 am

1288 bad sectors now! I wish I could find out where they were, ie. in which partitions.

I am short of external drive space for doing a backup just at the minute. I copied & compressed the NTFS bit, there were only 8GB of data there, but the computer won't let me copy it up to my NFS share, it has crashed twice doing it. The second time the screen just went blank with a cursor underline in the the top left. Just like a broken Beeb in fact!!

And it seemed to be using the hard drive 100% of the time (even though no internal hard drive access was needed for this operation) and using nearly 100% of CPU.

So that's not normal either. I wonder even if there might be a RAM fault as well.
Image

User avatar
richardtoohey
Posts: 3590
Joined: Thu Dec 29, 2011 5:13 am
Location: Tauranga, New Zealand
Contact:

Re: Linux partition format

Post by richardtoohey » Tue Oct 09, 2018 8:09 am

BeebMaster wrote:
Tue Oct 09, 2018 7:43 am
1288 bad sectors now! I wish I could find out where they were, ie. in which partitions. ... So that's not normal either.
Sounds like it is failing fast.

I'd recommend turning it off (or maybe just leaving it exactly as is - even an off/on again might be too much?) and leaving it until you've got something with enough space to copy everything across. And then fingers crossed it survives long enough to carry out the copy.

Or - if time was unlimited - learn how to mirror a drive and experiment with less important drives. Then once you know how, try and mirror the failing drive (i.e. at sector level).

You can then (after you've got the safe copy) poke around on the old drive trying to find out where the bad sectors are.

[-o< [-o< [-o<

User avatar
danielj
Posts: 6646
Joined: Thu Oct 02, 2008 4:51 pm
Location: Manchester
Contact:

Re: Linux partition format

Post by danielj » Tue Oct 09, 2018 8:12 am

Get a cheap external USB hdd and shift all your data over to that temporarily. The OS can just be downloaded again once you've got a new drive sorted out :) As Richard says, the location of the bad sectors isn't really relevant, just save the data!

Coeus
Posts: 1079
Joined: Mon Jul 25, 2016 11:05 am
Contact:

Re: Linux partition format

Post by Coeus » Tue Oct 09, 2018 1:40 pm

BeebMaster wrote:
Tue Oct 09, 2018 7:43 am
So that's not normal either. I wonder even if there might be a RAM fault as well.
That can obviously cause random crashes and corrupt the data being written to the disc. You can test with memtest86+. This should be an installable package or maybe available on a live CD/USB.

Also, given the concern about a failing HD, make sure you don't have swap space enabled on that disc as that can also lead to crashes/lock-ups.

ntfsclone can clone an NTFS partition into a file, copying only the used data.

VirtualBox VMs are in ~/VirtualBox VMs.
Last edited by Coeus on Tue Oct 09, 2018 1:44 pm, edited 1 time in total.

User avatar
BeebMaster
Posts: 2615
Joined: Sun Aug 02, 2009 4:59 pm
Location: Lost in the BeebVault!
Contact:

Re: Linux partition format

Post by BeebMaster » Tue Oct 09, 2018 9:31 pm

Oh yes, that memtest is in the grub menu I think.

Found my VMs, the total size of that dir is 151GB but I only have 85GB spare on my backup drive. However with one VM which I never use, and one file already a zip of a drive file the total goes down to 72GB so I am copying that now. I think as long as I get that done it won't be too tragic if the hard drive does die.

It's a "TOSHIBA MQ01ABD100 (AX0R5J)" 1000GB drive and it seems I can get the exact same model for only 40 quid, so that was a nice surprise, I was expecting a new hard drive to cost me nearly as much as the laptop did. A Seagate one, my favourite manufacturer, is about the same price but the dimensions are slightly different so it might rattle about a bit.

I don't think they do SSD drives that big yet, and although I don't really need a drive that big it makes sense to replace like for like for the remaining lifetime of this laptop.

I haven't replaced a laptop hard drive since Feb 1999 so I don't know how dismantlable this machine is. The other week I thought I would take the battery out and have a look at it, only to find there isn't one!! Must be built in hidden somewhere.
Image

Prime
Posts: 2579
Joined: Sun May 31, 2009 11:52 pm
Contact:

Re: Linux partition format

Post by Prime » Tue Oct 09, 2018 9:50 pm

BeebMaster wrote:
Tue Oct 09, 2018 9:31 pm
I haven't replaced a laptop hard drive since Feb 1999 so I don't know how dismantlable this machine is. The other week I thought I would take the battery out and have a look at it, only to find there isn't one!! Must be built in hidden somewhere.
Might be worth googling the model of the laptop + teardown or disassembily, that often yeilds useful results, often on youtube :)

My current Lenovo actually has 2 batteries, one internal fixed one, and one external swapable one, it can be powered off either it seems.

Cheers.

Phill.

User avatar
richardtoohey
Posts: 3590
Joined: Thu Dec 29, 2011 5:13 am
Location: Tauranga, New Zealand
Contact:

Re: Linux partition format

Post by richardtoohey » Wed Oct 10, 2018 6:31 am

BeebMaster wrote:
Tue Oct 09, 2018 9:31 pm
1000GB drive and it seems I can get the exact same model for only 40 quid ... I don't think they do SSD drives that big yet
They do, but it won't be 40 quid! And if the machine is that old, not sure about connectors etc. (but no doubt there'd be adaptors.)

User avatar
Elminster
Posts: 3137
Joined: Wed Jun 20, 2012 8:09 am
Location: Essex, UK
Contact:

Re: Linux partition format

Post by Elminster » Wed Oct 10, 2018 6:53 am

I just replace old harddisk in a 12 year old gateway laptop with an SSD. Luckily it is SATA, made a huge difference not just to speed, but noise and heat as well. Think I paid £40 for a 250gb model.

As Richard says TB is expensive, could have a small SSD and a larger external spinning harddisk, but would lose some of the benefits of the SSD.

Edit: De ja vu, I next read Colin’s harddisk thread :)
Last edited by Elminster on Wed Oct 10, 2018 6:55 am, edited 1 time in total.

User avatar
BeebMaster
Posts: 2615
Joined: Sun Aug 02, 2009 4:59 pm
Location: Lost in the BeebVault!
Contact:

Re: Linux partition format

Post by BeebMaster » Wed Oct 10, 2018 10:26 pm

I've ordered a new internal drive now, and an external USB one to do the backup/transfer. Should be here in a few days.

There's a lot of constant hard disc activity at the minute, but I don't know what's causing it. Usually it's either "tumbler" or "update-notifier", and killing either process stops it, but that hasn't worked. How do I identify which process is punishing my hard drive?
Image

User avatar
ctr
Posts: 183
Joined: Wed Jul 16, 2014 2:53 pm
Contact:

Re: Linux partition format

Post by ctr » Thu Oct 11, 2018 12:32 am

When my laptop's hdd was failing, reads of bad sectors were taking 30 seconds to time out. So even though there wasn't much going on the disk often appeared to be very busy.

You can use iotop to see what processes are using the disk. However, if it's not installed I wouldn't bother because installing it would produce another load of disk access.

User avatar
BeebMaster
Posts: 2615
Joined: Sun Aug 02, 2009 4:59 pm
Location: Lost in the BeebVault!
Contact:

Re: Linux partition format

Post by BeebMaster » Sun Oct 14, 2018 11:53 am

I installed iotop, it's very helpful. It was mainly tumbler, which doesn't seem to do anything useful, so I have uninstalled it. I did the same on a previous occasion a few years ago with no ill effects. I think when packages like that spend all their time smashing your hard drive, it's no wonder they start to develop errors.

It appears to have settled at 1288 bad sectors now. The external drive has arrived, but I'm still waiting for the new internal one.

In the meantime I used Clonezilla to replicate the existing drive. I didn't know till I had it running that it can write to a NFS share, I might have been better doing that. Also it doesn't seem to like encrypted drives for storing the image, so I had to move everything off the (nearly full) external drive to the new one so I could format the old one unencrypted and then use that. I also used to opportunity to move a lot of duplicated data in my /home off the laptop. In the end Clonezilla imaged the whole drive into about 86GB.

I left it running overnight, looking at access times it seems to have taken about 3 hours which is good going. It's something I should be doing periodically anyway I think.

On a related point - I need to free up a mini SD card for BeebSCSI. The only one I can spare has my unused Pi3 install on it. Can I just dd the card to make a complete image of it to restore later? Or do I need to explain the different partitions to it?

Clonezilla could do it, I'm sure, but I want to make the image whilst I'm running my usual OS.
Image

User avatar
BeebMaster
Posts: 2615
Joined: Sun Aug 02, 2009 4:59 pm
Location: Lost in the BeebVault!
Contact:

Re: Linux partition format

Post by BeebMaster » Sun Oct 14, 2018 12:07 pm

BeebMaster wrote:
Sun Oct 14, 2018 11:53 am
On a related point - I need to free up a mini SD card for BeebSCSI. The only one I can spare has my unused Pi3 install on it. Can I just dd the card to make a complete image of it to restore later? Or do I need to explain the different partitions to it?
To answer my own question, it appears to be "yes". I just did it, and I can mount the file by double-clicking on it, and the 3 partitions on the disc (PiBoot, PiRoot, Pi data) all show up as drives, although I can't actually access two of them as I get a "can't mount read only" error.
Image

User avatar
BeebMaster
Posts: 2615
Joined: Sun Aug 02, 2009 4:59 pm
Location: Lost in the BeebVault!
Contact:

Re: Linux partition format

Post by BeebMaster » Sat Oct 20, 2018 9:39 am

The new hard drive arrived earlier in the week. It's so light and tiny I thought the box was empty! Bring back full height MFM drives is what I say!

Anyhow, the existing drive is behaving at the minute. It's settled at 1288 bad sectors and it hasn't crashed for a while now. Rather than risk taking the laptop to pieces, I'm minded to keep it as it is for now, and make an image of the drive periodically, and only replace if disaster strikes or it gets noticeably worse.
Image

User avatar
Elminster
Posts: 3137
Joined: Wed Jun 20, 2012 8:09 am
Location: Essex, UK
Contact:

Re: Linux partition format

Post by Elminster » Sat Oct 20, 2018 10:18 am

Doesn’t the drive just slide out?, usually simple to replace drive on laptop.

You should find an ssd is quieter, cooler and runs much faster. Always the first mod I make older tired hardware.

Coeus
Posts: 1079
Joined: Mon Jul 25, 2016 11:05 am
Contact:

Re: Linux partition format

Post by Coeus » Sat Oct 20, 2018 1:19 pm

Elminster wrote:
Sat Oct 20, 2018 10:18 am
Doesn’t the drive just slide out?, usually simple to replace drive on laptop.
I depends on the laptop. "Business class" laptops often have removable covers or other easy ways to upgrade things - my work laptop has a slot for a HD/SSD, a bay that can contain an optical drive or another HDD and a cover under which to insert memory modules.

Consumer laptops are often a different kettle of fish. I upgraded the HD to a SSD in one recently and it involved taking the keyboard out, disconecting a bunch of stuff, taking the plam rest (top cover of the bottom half) out and then replacing the disc from the top. Much more fiddly,

User avatar
Elminster
Posts: 3137
Joined: Wed Jun 20, 2012 8:09 am
Location: Essex, UK
Contact:

Re: Linux partition format

Post by Elminster » Sat Oct 20, 2018 1:47 pm

To me they are all a piece of cake after swapping the drive out of my last iMac. That had to be the hardest drive I had to swap out. Sky box was a bit tricky, I managed to decspitated the power led when I swapped that one. Don’t remember ever having issues with laptops, maybe I am just lucky.

User avatar
CMcDougall
Posts: 6301
Joined: Wed Feb 02, 2005 3:13 pm
Location: Shadow in a Valley of Scotland
Contact:

Re: Linux partition format

Post by CMcDougall » Sat Oct 20, 2018 7:47 pm

Laptop HDiscs are easy! If they are normal not crap Apple.

If running out of space BeebMaster, if you have a non used TV vision box, they have 'laptop' 2.5" Sata drives in them to record Emmerdale :lol: my two up loft where 500gb , now in one laptop other in a Sony Playstation 3 :D

EDIT : cap to crap, but thought worse :mrgreen:
Last edited by CMcDougall on Sat Oct 20, 2018 7:48 pm, edited 1 time in total.
ImageImageImage

User avatar
BeebMaster
Posts: 2615
Joined: Sun Aug 02, 2009 4:59 pm
Location: Lost in the BeebVault!
Contact:

Re: Linux partition format

Post by BeebMaster » Tue Oct 23, 2018 7:20 pm

Crab apple maybe?

I did watch a video on how to take it to bits, there is a fair amount of dismantlage required. The last time I took a laptop apart I broke the ribbon connecting the touchpad, and also the ribbon for the CD drive. I don't know how I did break the CD one, it just seemed to get trapped when shutting the tray. So I don't think I'll replace the hard drive till it becomes "crucial".
Image

Prime
Posts: 2579
Joined: Sun May 31, 2009 11:52 pm
Contact:

Re: Linux partition format

Post by Prime » Tue Oct 23, 2018 9:42 pm

BeebMaster wrote:
Tue Oct 23, 2018 7:20 pm
Crab apple maybe?

I did watch a video on how to take it to bits, there is a fair amount of dismantlage required. The last time I took a laptop apart I broke the ribbon connecting the touchpad, and also the ribbon for the CD drive. I don't know how I did break the CD one, it just seemed to get trapped when shutting the tray. So I don't think I'll replace the hard drive till it becomes "crucial".
Or bring it to Leicester if you are coming, and let one of us loose on it.....

Cheers.

Phill.

Kazzie
Posts: 280
Joined: Sun Oct 15, 2017 7:10 pm
Location: North Wales
Contact:

Re: Linux partition format

Post by Kazzie » Wed Oct 24, 2018 7:34 pm

Prime wrote:
Tue Oct 09, 2018 9:50 pm
My current Lenovo actually has 2 batteries, one internal fixed one, and one external swapable one, it can be powered off either it seems.
My Lenovo x240 has an internal battery. This battery can be disabled via the BIOS before opening it up to avoid short circuits or similar. After being disabled, it won't be re-enabled until the mains power supply is attached.
BBC Model B 32k issue 7, Sidewise ROM board with 16K RAM
Archimedes 420/1 upgraded to 4MB RAM (mid- restoration)
Acorn System 1 home-made replica

User avatar
BeebMaster
Posts: 2615
Joined: Sun Aug 02, 2009 4:59 pm
Location: Lost in the BeebVault!
Contact:

Re: Linux partition format

Post by BeebMaster » Thu Oct 25, 2018 7:33 pm

It had another paddy last night, and went to 1640 bad sectors, so I thought I'd better do the deed. I patched it up with fsck and then this morning I used Clonezilla to make another clone of the drive.

Tonight I took it apart, replaced the drive, didn't break anything, reassembled it, and restored the clone. All in all took just over an hour. I thought grub was going to be a problem needing rebuilding, as Clonezilla said it was restoring to /dev/sdb what it had copied from /dev/sdc, and this might render grub up the spout, and of course it should really be /dev/sda anyway, but I'd plugged the Clonezilla drive in and the backup drive before powering it on, so the drives were detected in the wrong order.

Anyway, none of that came to pass, it booted up to my usual login screen first time. So here I am, with 0 bad sectors! Good old Seagate, they have always been my favourite.

Thanks to Phill for showing me Clonezilla, it would have been a nightmare without that, and to everyone who helped.
Image

Prime
Posts: 2579
Joined: Sun May 31, 2009 11:52 pm
Contact:

Re: Linux partition format

Post by Prime » Thu Oct 25, 2018 9:58 pm

BeebMaster wrote:
Thu Oct 25, 2018 7:33 pm
Thanks to Phill for showing me Clonezilla, it would have been a nightmare without that, and to everyone who helped.
Glad to be of help, and glad it went ok.

Clonezilla may have it's rough edges, as it won't shrink partitions, only expand them but it's certainly made my life easier both at work and at home.

Cheers.

Phill.

Post Reply