Set the oven to 170°C (fan) / 190°C / gas mark 5. Grease and line a 2lb loaf tin with baking paper.
Add the softened butter and olive oil to the sugar and beat until light and fluffy. Mix in the eggs one at a time. Stir in the lemon juice and lavender buds.
In a separate bowl, whisk together the flour, baking powder and salt.
Add half the dry mix to the wet ingredients and gently fold in. Add the yoghurt, then fold in the remaining dry ingredients until just combined. Don’t overmix.
Pour the batter into your prepared tin and smooth the top. Bake for 45–55 minutes until a skewer inserted into the centre comes out clean. Baking times vary depending on your oven. Cover loosely with foil if it starts to brown too quickly - lemon cakes do tend to brown, this is OK!
Let the cake cool in the tin for 10–15 minutes before turning out onto a wire rack. Mix the icing sugar with just enough lemon juice to make a pourable glaze, then drizzle over the cooled cake. Top with lavender or zest if using.