Building a LinkedIn Data Integration with Next.js
Tech Stack: Next.js, TypeScript, LinkedIn API, Prisma, PostgreSQL
Building a LinkedIn Data Integration with Next.js
When building my portfolio website, I wanted to showcase my professional experience and recommendations from LinkedIn without manually updating content. After exploring various options, I decided to integrate with LinkedIn's Data Member Access (DMA) API. This journey taught me valuable lessons about API integration, OAuth authentication, and intelligent data caching strategies.
1. Understanding LinkedIn's API Landscape
LinkedIn offers several APIs, but for accessing member profile data, the Data Member Access API is the way to go. This API is part of LinkedIn's compliance offerings and requires specific OAuth scopes:
- DMA API: Provides access to member profile data exports
- OAuth 2.0: Uses client credentials flow for authentication
- Required Scope:
r_dma_portability_self_serve - API Version: Specified via the
LinkedIn-Versionheader (e.g., "202312")
The key insight here is that LinkedIn's DMA API works differently from traditional REST APIs. Instead of instant responses, it follows an asynchronous export pattern where you request data and then poll for completion.
2. Setting Up OAuth 2.0 Authentication
The first step was setting up OAuth authentication with LinkedIn. Here's what's required:
# Environment variables needed
LINKEDIN_CLIENT_ID=your-client-id
LINKEDIN_CLIENT_SECRET=your-client-secret
LINKEDIN_ACCESS_TOKEN=your-access-token
LINKEDIN_API_VERSION=202312
LINKEDIN_SCOPE=r_dma_portability_self_serve
To obtain an access token, you'll need to:
- Register your application on LinkedIn Developer Portal
- Request access to the DMA API (requires compliance review)
- Use the client credentials flow to obtain an access token
- Store the token securely in your environment variables
Microsoft provides comprehensive documentation on the DMA API implementation.
3. Implementing the LinkedIn Service
The core of my implementation is a TypeScript service class that handles all LinkedIn API interactions. Here's the key architecture:
class LinkedInService {
private readonly accessToken: string;
private readonly apiVersion: string;
private rateLimitCounter: number;
private rateLimitResetTime: number;
constructor() {
this.accessToken = process.env.LINKEDIN_ACCESS_TOKEN!;
this.apiVersion = process.env.LINKEDIN_API_VERSION || '202312';
this.rateLimitCounter = 0;
this.rateLimitResetTime = Date.now() + 24 * 60 * 60 * 1000;
}
async getProfile(): Promise<LinkedInProfile> {
// Implementation details...
}
}
The service handles several critical aspects:
- Rate limiting: LinkedIn has a 200 requests per 24-hour limit
- Retry logic: Implements exponential backoff for failed requests
- Error handling: Gracefully handles API errors and falls back to cached data
4. The Data Fetching Flow
LinkedIn's DMA API has a unique flow that differs from typical REST APIs:
// 1. Check member authorization
const isAuthorized = await this.checkMemberAuthorization();
// 2. If not authorized, register for DMA
if (!isAuthorized) {
await this.registerForDMA();
}
// 3. Request data export for specific domains
const domains = ['PROFILE', 'POSITIONS', 'SKILLS', 'EDUCATION', 'RECOMMENDATIONS'];
const snapshotData = await this.fetchAllSnapshotData(domains);
// 4. Format and return the data
return this.formatProfileFromData(snapshotData);
The asynchronous nature means you need to handle polling and status checking, which adds complexity but ensures data completeness.
5. Intelligent Database Caching
To minimize API calls and improve performance, I implemented a PostgreSQL caching layer using Prisma:
async getProfile(): Promise<LinkedInProfile> {
// Check database for existing profile
const existingProfile = await prisma.linkedInProfile.findFirst({
where: { id: 1 },
include: {
experiences: true,
skills: true,
recommendations: true,
// ... other relations
}
});
// Use cached data if less than 24 hours old
if (existingProfile && !isOutdated(existingProfile.lastUpdated)) {
return formatProfileFromDB(existingProfile);
}
// Otherwise, fetch from LinkedIn API
const freshData = await this.fetchFromLinkedInAPI();
await this.saveProfileToDatabase(freshData);
return freshData;
}
This caching strategy provides several benefits:
- Reduced API calls: Stay well within rate limits
- Faster page loads: Database queries are much faster than API calls
- Reliability: Fallback to cached data if API is unavailable
- Cost efficiency: Fewer external API calls mean lower operational costs
6. Handling Pagination for Large Data Sets
One challenge I encountered was with recommendations - they often span multiple pages. Here's how I handled pagination:
private async fetchDomainWithPagination(domain: string): Promise<any[]> {
let allItems: any[] = [];
let nextUrl = '/memberSnapshotData?q=criteria&domain=' + domain;
while (nextUrl) {
const data = await this.fetchWithRetry(nextUrl);
// Extract items from current page
if (data.elements?.length > 0) {
const element = data.elements.find(el => el.snapshotDomain === domain);
if (element?.snapshotData) {
allItems = [...allItems, ...element.snapshotData];
}
}
// Check for next page
nextUrl = this.getNextPageUrl(data.paging);
}
return allItems;
}
This pagination approach ensures I capture all recommendations, not just the first page.
7. Data Transformation and Type Safety
LinkedIn's API returns data in a specific format that needs transformation for UI consumption:
interface LinkedInProfile {
title: string;
about: string;
experience: Experience[];
education: Education[];
skills: Skill[];
recommendations: {
received: Recommendation[];
given: Recommendation[];
};
courses: Course[];
}
// Transform raw API data to typed interfaces
private formatExperience(data: any[]): Experience[] {
return data.map(item => ({
title: item['Title'] || '',
companyName: item['Company Name'] || '',
location: item['Location'] || '',
startDate: item['Started On'] || '',
endDate: item['Finished On'] || undefined,
description: item['Description'] || ''
}));
}
TypeScript interfaces ensure type safety throughout the application, preventing runtime errors and improving developer experience.
8. Displaying Data in Next.js Components
With the data fetched and cached, displaying it in Next.js components is straightforward:
// Server Component
export default async function ProjectsPage() {
const profile = await linkedInService.getProfile();
return (
<div>
<ProfileSection profile={profile} />
<ExperienceSection experiences={profile.experience} />
<SkillsSection skills={profile.skills} />
<RecommendationsSection recommendations={profile.recommendations} />
</div>
);
}
The beauty of this approach is that the components remain simple and focused on presentation, while the complex data fetching logic is abstracted away.
9. Admin Interface for Data Management
I built an admin interface to manage the LinkedIn integration:
export default function LinkedInAdmin() {
const [isUpdating, setIsUpdating] = useState(false);
const handleForceUpdate = async () => {
setIsUpdating(true);
try {
await fetch('/api/linkedin/force-update', { method: 'POST' });
toast.success('LinkedIn data updated successfully');
} catch (error) {
toast.error('Failed to update LinkedIn data');
} finally {
setIsUpdating(false);
}
};
return (
<div>
<h2>LinkedIn Data Management</h2>
<Button onClick={handleForceUpdate} disabled={isUpdating}>
{isUpdating ? 'Updating...' : 'Force Update'}
</Button>
</div>
);
}
This interface allows manual data refresh when needed, useful for development and ensuring fresh data before important presentations.
10. Lessons Learned and Best Practices
Through this implementation, I learned several valuable lessons:
Rate Limiting Strategy
- Implement a counter that resets every 24 hours
- Leave a buffer (e.g., use only 180 of 200 allowed requests)
- Always check rate limits before making requests
Error Handling
- Implement retry logic with exponential backoff
- Fall back to cached data when API is unavailable
- Log errors comprehensively for debugging
Data Freshness
- Balance between data freshness and API usage
- 24-hour cache duration works well for profile data
- Provide manual refresh option for critical updates
Security Considerations
- Never expose API credentials in client-side code
- Use environment variables for all sensitive data
- Implement proper authentication for admin endpoints
Conclusion
Integrating LinkedIn's DMA API into my Next.js portfolio has been a rewarding experience. The combination of intelligent caching, type-safe data handling, and robust error management creates a reliable system that showcases professional data without manual maintenance.
The key to success was understanding LinkedIn's unique API patterns and building abstractions that handle the complexity while providing a simple interface for the rest of the application.
For more details on LinkedIn's DMA API, check out:
If you're building a similar integration, remember to focus on reliability over real-time updates - your users will appreciate a fast, stable experience over always-fresh data that's slow to load.