I believe there is a direct train from Yate/Bristol Parkway/Temple Meads.. Checked the current timetable and there is at least one direct train during weekdays but not weekends.
You can also travel to Southampton by rail quite easily and change there for Brighton, although it would be less stressful to go the day before and stay over as suggested by
@vernon.