Ranger admin web 模块分析之policy查询缓存

ranger admin web 模块分析之policy查询缓存 admin web模块

开放的api 在类PublicAPIsv2中，这个类中的方法都是实时查询的。主要API涉及

• ServiceDef Manipulation APIs
• Service Manipulation APIs
• Policy Manipulation APIs。

管理端WebUI的查询policy走ServiceREST中的getServicePolicies方法。

    @GET
    @Path("/policies/service/{id}")
    @Produces({"application/json", "application/xml"})
    public RangerPolicyList getServicePolicies(@PathParam("id") Long serviceId,
                                               @Context HttpServletRequest request) {
   ...
   ...
   ...
}

根据 传入的条件参数构造SearchFilter对象，如果 isAdminUserWithNoFilterParams(filter)，则通过svcStore.getPaginatedServicePolicies(serviceId, filter)分页查询，通过ServiceDBStore对象的getServicePolicies查询XXService下的policies，其中通过缓存类RangerServicePoliciesCache下的getServicePolicies方法获取policy。

如果不采用缓存 会调用ServiceREST的getServicePolicies方法。

SearchFilter中的成员属性,默认排序是以id的asc进行

private Map params;
private int                 startIndex;// 开始下标
private int                 maxRows    = Integer.MAX_VALUE; // 最大行数
private boolean             getCount   = true; // 是否计算总数
private String              sortBy; // 排序字段 只有（createTime,updateTime,policyId,policyName）
private String              sortType;//排序类型 asc desc

RangerServicePoliciesCache类详解

该类主要用于查询和缓存各Service的所有Policies。RangerServicePoliciesCache采用单例模式进行实例化，它的私有构造函数中会加载ranger.admin.policy.download.usecache和ranger.admin.policy.download.cache.max.waittime.for.update两个配置数据

第一个代表是否采用缓存 默认开启

第二个是缓存getLast的锁等待时间Update的时间默认10s。

主要方法getServicePolicies(String serviceName, Long serviceId, ServiceStore serviceStore)

	public ServicePolicies getServicePolicies(String serviceName, Long serviceId, ServiceStore serviceStore) throws Exception {

		if (LOG.isDebugEnabled()) {
			LOG.debug("==> RangerServicePoliciesCache.getServicePolicies(" + serviceName + ", " + serviceId + ")");
		}

		ServicePolicies ret = null;

		if (StringUtils.isNotBlank(serviceName) && serviceId != null) {

			if (LOG.isDebugEnabled()) {
				LOG.debug("useServicePoliciesCache=" + useServicePoliciesCache);
			}

			ServicePolicies servicePolicies = null;
            // 是否开启缓存的使用
			if (!useServicePoliciesCache) {
				if (serviceStore != null) {
					try {
                        //方法体中正在去查询police的是ServiceDBStore的List getServicePoliciesFromDb(XXService service)
						servicePolicies = serviceStore.getServicePolicies(serviceName);
					} catch (Exception exception) {
						LOG.error("getServicePolicies(" + serviceName + "): failed to get latest policies from service-store", exception);
					}
				} else {
					LOG.error("getServicePolicies(" + serviceName + "): failed to get latest policies as service-store is null!");
				}
			} else {
				ServicePoliciesWrapper servicePoliciesWrapper = null;
                // 代码块加锁
				synchronized (this) {
					servicePoliciesWrapper = servicePoliciesMap.get(serviceName);

					if (servicePoliciesWrapper != null) {
						if (!serviceId.equals(servicePoliciesWrapper.getServiceId())) {
							if (LOG.isDebugEnabled()) {
								LOG.debug("Service [" + serviceName + "] changed service-id from " + servicePoliciesWrapper.getServiceId()
										+ " to " + serviceId);
								LOG.debug("Recreating servicePoliciesWrapper for serviceName [" + serviceName + "]");
							}
							servicePoliciesMap.remove(serviceName);
							servicePoliciesWrapper = null;
						}
					}

					if (servicePoliciesWrapper == null) {
						servicePoliciesWrapper = new ServicePoliciesWrapper(serviceId);
						servicePoliciesMap.put(serviceName, servicePoliciesWrapper);
					}
				}

				if (serviceStore != null) {
                    // 从缓存中获取policy
					boolean refreshed = servicePoliciesWrapper.getLatestOrCached(serviceName, serviceStore);

					if(LOG.isDebugEnabled()) {
						LOG.debug("getLatestOrCached returned " + refreshed);
					}
				} else {
					LOG.error("getServicePolicies(" + serviceName + "): failed to get latest policies as service-store is null!");
				}

				servicePolicies = servicePoliciesWrapper.getServicePolicies();
			}

			ret = servicePolicies;

		} else {
			LOG.error("getServicePolicies() failed to get policies as serviceName is null or blank and/or serviceId is null!");
		}

		if (LOG.isDebugEnabled()) {
			LOG.debug("