Go to Greatfish Isle first by southwest two islands apart. Then south three islands apart and watch out for danger along the way. Notice while you sail to your island that daylight hasn't arrived yet? The curse has been put on so you don't have to worry about Tetra's ship.
Before you go behind the island to take down the stone slab, you will need two things, 1000 rupee wallet and an elixir from your grandmother. So go to the top of the mountain, climb the highest stone, point the wind to west and go in to the area where you first meet Tetra. Instead of going to where Moblins were, bomb the big stone rock and go in to meet the Giant Fairy. You'll acquire and upgraded wallet which will be upgraded again soon. Also get the fairy in an empty bottle. Go back to heal your Grandma with the fairy and she'll reward your with elixir soup. Because she is so kind, you get to drink it twice and you'll only get one so if it's empty, you can go back to grandma to get some more.
Once you're finished with this sidequests, go back to the King of Red Lions boat and cruise to where the whirpool is. There's three parts to it and it takes two bomb hits until you break down the stone slab. Jabun will be meeting you and King of Red Lions and gives the Nayru's Pearl to you. That ends the Ganon curse believing it will slow you down.